A thousand tools, one small backpack

Imagine you are going to school and you own a thousand tools. A hammer, a glue stick, a microscope, a trumpet, forty kinds of screwdriver. You also own two thousand recipe cards that tell you how to do things: how to build a birdhouse, how to file taxes, how to fix a leaking tap.

Your backpack holds about twenty things.

Every single morning you have to decide what goes in. Whatever you leave in the warehouse is not immediately available unless you carry a catalog and fetch it later. Whatever you put in the bag, you carry all day even if you never touch it. And the bus driver only lets you skip the security line if the front of your bag looks exactly the same as yesterday.

That is the problem every coding agent harness hits once it grows up. This post places four approaches beside the same absurdly large company, hands each the same incident, and watches what reaches the model: Codex, Claude Code, Hermes Agent, and DeepSeek Harness.

The company and counts below are a capacity thought experiment, not measured production cardinalities or a benchmark. A harness can have a huge registered universe; the question is:

The backpack is the context window

A language model sees exactly one thing: this request. The request has three compartments.

• The system prompt. Who you are, how to behave. Ideally small and stable.
• The tool schemas. A JSON description of every tool the model may call. Name, description, parameters.
• The messages. The conversation so far, including every tool result.

Here is the part kids and engineers both miss the first time. Conversation history grows as a suffix and can later be compacted, spilled, or represented by provider continuation state. An eagerly exposed tool schema remains in the callable request surface every turn, whether the tool gets used or not. It is a recurring tax.

✕

Two more facts make it worse. Agent runs are input-heavy because instructions, tool definitions, and retained history accompany many generations. And provider prompt caches reward stable prefixes: changing a tool definition can move the reusable boundary and force more input to be processed again. Exact cache accounting varies by provider, but the design pressure is the same.

So the question is not “can we fit a thousand tools?” It is “how do we own a thousand tools and carry twenty, without forgetting the other nine hundred and eighty exist?”

Two ways to pack badly

There are exactly two ways to get this wrong, and every design in this post is a way to stand between them.

The harness studies behind this article establish the mechanisms below, not universal success rates at 50,000 tools. Search makes selection possible; it does not prove that a given model will retrieve the right operation. Large-catalog recall, first-call argument validity, payload size, cache behavior, and end-to-end task success still need model- and provider-specific evaluation.

Five things that look the same and are not

Before the harnesses, a distinction they all make and most teams do not. There are five different objects in the bag, and each pays a different tax.

1. SYSTEM PROMPT        "How should you behave?"
   coding rules, safety, tool-use instructions, environment, policies

2. CORE TOOLS           "Things you can always do"
   shell, read_file, edit_file, search — the hammer and the pencil

3. SKILLS               "Recipes"
   how to investigate an incident, how to review a PR
   a SKILL.md file with steps, references, maybe scripts

4. MCP TOOLS            "Borrowed tools"
   get_logs(), restart_deployment(), create_clickup_task()
   thousands of them, owned by other people's servers

5. PLUGINS              "Boxes of the above"
   one plugin can ship skills + MCP connections + tools + commands + hooks

Two rules fall out of this list.

MCP tools are tools. They share the schema compartment with core tools. If you flatten every neighbor’s garage into the bag, you have already lost.

Skills are not tools. They do not go in the schema compartment at all. They are documents the model reads when a task matches. Every serious implementation says this out loud. Confuse the two and you pay the wrong tax.

And a plugin is not one tool either. An “Engineering Operations” plugin might add three skills, two MCP servers, fifteen native tools, and four commands. Installing it grows the registered universe. It should not grow what the model sees on every turn. Each thing inside it should fall into its own catalog mechanism.

Meet the monster company

Now let’s build the hypothetical environment we will use to compare the four approaches. It is deliberately oversized so the architectural limits are impossible to ignore.

Agent installation
│
├── Core tools                          ~50
│   terminal · read_file · edit_file · web_search · browser · …
│
├── 250 Skills
│   incident-response · kubernetes-debugging · github-pr
│   postgres-performance · customer-support · …
│
├── 150 Plugins                         ~500 tools between them
│   engineering · support · sales · …
│
└── 100 MCP servers                     ~50,000 tools
    ├── CRM MCP              1,400 tools
    ├── AWS MCP              1,100 tools
    ├── Kubernetes MCP         850 tools
    ├── Datadog MCP            600 tools
    ├── ClickUp MCP            300 tools
    ├── GitHub MCP             220 tools
    ├── Slack MCP              180 tools
    └── … 94 more

At a modest 500 tokens per schema, putting all 50,000 MCP tools in the bag would cost 25 million tokens per request. Not “expensive”. Impossible. So every one of these harnesses must do something. The interesting part is what.

Here is the task the user hands to each of them:

To finish this, the agent needs roughly:

incident-response skill          (how to do it)
core repo/file/terminal tools    (inspect the code change)
Datadog MCP                      (production errors, 429s)
Kubernetes MCP                   (rollout history, pod state)
GitHub MCP                       (the PR, maybe)
ClickUp MCP                      (create the incident)
Slack MCP                        (post the summary)

Seven capability sources. Five of them are MCP servers with hundreds of operations each. The agent needs about one operation from each. The comparison starts with a 50,000-tool registry, but not every provider can admit that whole registry into one request; some need preselection or partitioning first.

The shared principle: the warehouse is not the backpack

All four separate the registered universe from the surface the model must reason over. They do not, however, use the same wire protocol or pay the same remaining cost.

The useful ideal is a card catalog: keep compact routing hints, find the relevant item, and activate only what the task needs. Codex can append a typed schema-bearing load item. Claude keeps an admitted deferred subset on the wire and returns a compact reference. Hermes returns metadata and later a schema in ordinary tool history. DeepSeek-style Code Mode has no equivalent search primitive in the studied path: it describes every already-selected tool in a generated SDK.

✕

Notice there are two separate card catalogs in that picture, and it matters.

"Which TOOL?"        → capability discovery → deferred tools
"Which PROCEDURE?"   → skill disclosure     → SKILL.md body

A skill tells the agent how to run an incident. It does not itself fetch a single log line. The tools do that. Skills route procedure; tools route capability. Every harness below keeps those two axes apart.

The four harnesses differ in four places:

1. What goes on a card, and how many cards fit.
2. How the model looks a card up.
3. What comes back, and where it lands.
4. What survives when the bag gets emptied by compaction.

Let us run the incident through each.

Codex: registered is not exposed

Codex starts with one useful abstraction. Registered tools can be exposed as DIRECT, DEFERRED, or CODE_MODE, and those modes can combine. A tool being available is a separate fact from its schema being model-visible on this turn. Runtime dispatch stays registered even when the schema is deferred.

Here is what the first request looks like in our monster company:

MODEL REQUEST #1 (Codex)

SYSTEM PROMPT            you are Codex, coding behavior, tool rules, permissions
PROJECT CONTEXT          AGENTS.md, working directory
SKILL CATALOG            incident-response · kubernetes-debugging · github-pr · …
                         (name + one-line description + locator, each)
DIRECT TOOLS             shell · read_file · apply_patch · …
DEFERRED MECHANISM       tool_search  +  "these namespaces exist: k8s, datadog, …"

NOT PRESENT              ❌ 50,000 MCP schemas

Direct is the front pocket: small, common, full schemas. Why would you ever defer read_file? It is cheap and used constantly.

Deferred is where MCP and app tools go when the model and provider support it. The first request carries a tool_search tool but none of the deferred schemas. When the model searches, Codex runs an in-memory BM25 index over tool names, namespaces, descriptions, parameter names, and nested schema descriptions. Underscores become spaces so get_rollout_history matches “rollout history”.

Now the important precision. Search does not paste a new definition into the top-level tools array, but the studied client-search path does return a typed tool_search_output containing the matched namespace and full function schema with defer_loading: true. The integration test checks the sequence: request one has no deferred Calendar schema; Codex runs BM25 locally; request two carries the matched schema inside structured history; later top-level tools arrays remain unchanged; and the provider emits the namespaced exact call.

In backpack terms: the librarian hands you the exact relevant page in a protocol envelope instead of repacking the front pocket. The stable tool prefix is the saving. This does not mean zero post-search context, history, or wire cost: the matched schema can still cross the wire in that typed item.

For visibility, deferred namespaces still get a cheap advertisement: a name and the first line of description, under a 4 KiB total budget and 250 characters per namespace, with incremental add and remove diffs after the first snapshot. MCP server instructions become routing hints, which is why the official guidance says to make the first 512 characters self-contained.

Codex runs the incident

USER   "Voice AI calls started failing after yesterday's deploy…"

MODEL  reads skill catalog → incident-response matches
       reads SKILL.md from disk           ← body arrives now, bounded to 8 KB
       procedure: timeline → deploy → app errors → infra → provider → record → notify

MODEL  git log · read deployment config · grep "VoiceAI"
                                          ← DIRECT tools, no discovery

MODEL  tool_search("kubernetes deployment rollout pods")
       → typed load item: kubernetes.get_rollout_history, kubernetes.list_pods
       kubernetes.get_rollout_history({namespace:"prod", deployment:"voice-ai"})

MODEL  tool_search("production error rate 429 metrics")
       → typed load item: datadog.query_metrics, datadog.query_logs
       datadog.query_logs({...})

MODEL  tool_search("create ClickUp task")     → clickup.create_task(...)
MODEL  tool_search("post slack channel message") → slack.post_message(...)

Four searches, four typed matches, four exact calls. The top-level tool prefix never changed. Fifty thousand tools were registered; only the matches entered the model’s effective tool context.

Skills get a bounded catalog rather than a search. Each visible skill renders as a name, a routing description, and a locator, which is a filesystem path or a resource id. The budget is 2% of the context window when known, else 8,000 characters, capped at 10,000 tokens, with 1,024 characters per description. When the catalog does not fit, Codex keeps every name-and-locator line first and hands out remaining description characters round-robin. If even the names do not fit, tail entries drop and a warning fires.

There are BM25 skill selectors in the tree, but their interface says they run in shadow mode without changing the model-visible catalog. They are evaluation infrastructure, not routing.

Compaction restores rather than summarizes the durable layers. Old history is summarized, recent user messages are kept up to a 20,000-token budget, and the initial context, including repo instructions and the skill catalog, is re-injected around the latest real user request.

Where Codex still falls down: on a provider that cannot represent namespaced search results, MCP tools fall back to eager direct exposure, and you are back at the original problem. And our 250 skills are fine, but at thousands of long locators the tail can still vanish.

Claude Code: deferred by default, and big results go to disk

Claude Code lands in the same place as Codex with a slightly different route, because it can lean on the Anthropic API directly.

On supported models, MCP tools are deferred by default. At session start the model sees bounded routing guidance rather than every full schema. Schemas load when a ToolSearch call finds them. The protocol underneath:

1. Admitted candidate definitions stay in the API tools request with defer_loading: true.
2. Deferred definitions are excluded from the model’s system-prompt prefix.
3. ToolSearch returns tool_reference blocks for matches.
4. The API expands those references for the model.
5. The search result stays in history, so the tool can be reused later without another search.

The result resembles Codex at a distance, but its wire shape is different. Because Claude’s admitted deferred definitions still travel in the API request, this is primarily a model-context and prompt-prefix win, not a wire-bytes win. Those are two different costs:

bytes serialized over the wire   ≠   tokens the model attends to

Claude Code’s deferral is strong at the second, which is the one that decides both accuracy and the bill.

The native API accepts at most 10,000 deferred definitions in one request, and request bytes may become the binding constraint earlier. A 50,000-tool authorized universe therefore has to be prefiltered or partitioned before Claude’s ToolSearch can search it. The public docs do not establish the proprietary runtime’s exact ranking algorithm, so the honest claim is the protocol, not a particular ranker.

Claude Code runs the incident

registered universe = 50,000 tools
request projection  = authorized, ranked/partitioned subset (≤10,000; often fewer)
tools on the wire    = [ Read, Edit, Bash, Glob, Grep, …, ToolSearch ]
                     + admitted definitions marked defer_loading
model's first view   = eager tools + search + bounded routing guidance

MODEL  skill catalog → incident-response → SKILL.md injected as one message
MODEL  Read · Grep · Bash(git log)            ← core, eager
MODEL  ToolSearch("kubernetes deployment rollout history")
       → tool_reference: mcp__k8s__get_rollout_history
       mcp__k8s__get_rollout_history({...})
MODEL  ToolSearch("production 429 error metrics voice ai")
       → mcp__datadog__query_logs, mcp__datadog__query_metrics
MODEL  ToolSearch("create ClickUp task")  → mcp__clickup__create_task
MODEL  ToolSearch("post Slack message")   → mcp__slack__post_message

Three turns later, the retained Datadog reference can be reused without another search. That is an optimization, not durable capability truth: compaction, failover, or lost continuation state may require reconstructing provider state from separately persisted capability IDs, revisions, and digests.

What Claude Code adds is operational dials. The ENABLE_TOOL_SEARCH=auto setting loads everything eagerly while total schemas are under 10% of the window and defers at or above it. You can set a custom percentage, turn it off, or mark a small hot server alwaysLoad so three to five universal tools stay direct while the long tail defers. A non-first-party base URL disables search by default because many proxies drop reference blocks.

The second thing it adds is a rule for big tool results. Suppose Datadog returns seventeen megabytes of logs. If that sits in the chat forever, the conversation dies. Claude Code warns above 10,000 MCP-output tokens and caps at 25,000 by default. Beyond the threshold, text results are written to disk and replaced in the conversation by a file reference the model can read or grep later. The durable artifact leaves the prompt; a recoverable pointer stays. That is the result lifecycle side of scaling, and it is as important as discovery.

Skills follow metadata-first, body-later. Names and descriptions load at startup under a 1% window budget. Every name is retained; descriptions are removed starting with the least-invoked skills, and each description is capped at 1,536 characters. Invoking a skill injects the rendered SKILL.md as a single conversation message. A repeat invocation produces a short “already loaded” note instead of a second copy. disable-model-invocation: true hides a manual-only skill from the catalog entirely, and context: fork runs a skill in a fresh subagent and returns only a summary.

After compaction, the system prompt is unchanged, CLAUDE.md and memory are re-read from disk, up to five recently modified files are re-read, and invoked skills are re-attached at up to 5,000 tokens each and 25,000 total, newest first.

Where Claude Code still falls down: the skill catalog degrades to name-only gracefully, but there is no documented search gate for skill metadata. At thousands of skills, capability is visible and practically undiscoverable unless names are excellent.

Hermes Agent: what if the model has never heard of a reference?

Hermes asks a question the other two do not have to. What if the model is Qwen, DeepSeek, GLM, Llama, or something on OpenRouter? Those models understand ordinary function calling. They have no idea what an Anthropic tool_reference or an OpenAI namespaced deferred tool is.

So Hermes builds its own portable protocol out of plain functions, and states two invariants in its contributor guide that this whole post is about:

1. Per-conversation prompt caching is sacred.
2. The core is a narrow waist. Every core tool is paid on every API call.

The studied tree contains a broad set of roughly 59 core tools. Core tools are normally eager, but explicit and default defer sets take precedence—session_search and todo_list, for example, are default-deferred in the studied source. Optional tools can also carry a check_fn that omits them from the schema until a prerequisite exists, such as a token or a platform. Zero footprint otherwise.

At startup Hermes connects to each MCP server, calls list_tools(), and registers everything under prefixed names like mcp_github_create_issue, so five servers that each have a search tool do not collide. In our company that is about 50,000 registrations. None of their schemas go on the wire. Instead the model sees three bridge tools:

HERMES MODEL SEES

normally eager core  terminal · read_file · patch · web_search · …
bridge tools (3)     tool_search · tool_describe · tool_call

behind the curtain   50,000 deferred tools

Any ordinary function-calling provider can represent those three. Whether a particular model follows the search–describe–call protocol reliably is still an evaluation question. That is the portability bet and its cost.

The piece worth copying is how Hermes keeps advertisement bounded. Inside the tool_search description, under a budget of min(5% of context, 4,000 tokens), it degrades from names plus descriptions to names only, mixed per-source detail, source summaries, and finally no flat listing while the search control remains:

✕

At 50,000 tools, the flat catalog may degrade to source summaries or disappear altogether. The model still sees the search control, and may see hints such as “kubernetes: 850 tools,” but the implementation does not guarantee that every source or operation remains advertised. Hermes proves a bounded operating posture; it does not prove retrieval accuracy at 50,000 or 100,000 tools.

Search is BM25 over name, server, description, and top-level parameter names, with a substring fallback. The catalog is rebuilt on every assembly rather than cached per session, and it is scoped to the session’s toolset, so a subagent cannot search outside what it was granted.

Hermes runs the incident

MODEL  skills_list → incident-response
       skill_view("incident-response")          ← body arrives as a tool result

MODEL  read_file · search_files · terminal       ← core, always present

MODEL  tool_search(["kubernetes deployment rollout history"])
       → mcp_kubernetes_get_rollout_history, mcp_kubernetes_list_pods

MODEL  tool_describe(["mcp_kubernetes_get_rollout_history"])
       → { namespace: string, deployment: string, revision_limit?: number }

MODEL  tool_call(name="mcp_kubernetes_get_rollout_history",
                 arguments={namespace:"prod", deployment:"voice-ai"})

       HERMES BRIDGE resolves the real tool → MCP client → Kubernetes MCP
       hooks, guardrails, approvals run against the real tool, not the bridge

MODEL  search → describe → call   for datadog
MODEL  search → describe → call   for clickup
MODEL  search → describe → call   for slack

Notice the extra step. Codex goes search, typed schema-bearing load, exact call; Claude Code goes search, compact reference, exact call. Hermes goes search, describe, generic call, and the harness unwraps tool_call into the real operation. The model never receives Kubernetes as a first-class provider tool. We will come back to why that matters.

Hermes deliberately skips Code Mode, citing “large surface area”.

Skills live in a separate catalog. The index occupies the volatile tier of the system prompt with short descriptions, while the body is loaded by skill_view and arrives as a tool result. After compaction a [SKILL_PRUNED] marker tells the model to reload. As with tool advertising, bounded metadata can degrade, so good names and explicit reload behavior matter.

Plugins in Hermes can register tools, hooks, slash commands, and skills, and can call configured MCP servers. Non-core plugin tools join the deferred catalog alongside MCP tools. Core stays eager.

The system prompt is built once per session and rebuilt only on compression, in three tiers: stable identity and tool guidance, then repo context and workspace snapshot, then volatile skills index and memory with a date-only timestamp. Micro-compaction is off by default because it would break the cache every turn.

Finally, the Footprint Ladder. When someone wants to add a capability, the contributor guide says to try each rung in order and stop at the first that works:

1. Extend existing code.
2. A CLI command plus a skill. Zero schema.
3. A service-gated tool with check_fn.
4. A plugin.
5. An MCP server in the catalog.
6. A new core tool. Last resort.

That ladder is a cultural answer to a technical problem. Most capability sprawl is people reaching for rung six when rung two would do.

DeepSeek Harness: one door after selection

DeepSeek Harness makes everything a plugin, including the tool registry, the system-prompt assembler, skills, MCP, and compaction. Code Mode changes how a bounded tool set is presented and composed; it does not solve catalog selection by itself.

Tools have a presentation mode: native, code, or both. In code, the top-level callable list collapses to one reserved tool, run_code, but a generated SDK section still contains the input and output signatures for every selected visible tool. That prompt remains O(N), so a 50,000-tool registry must be narrowed before generation. The model writes a script against the selected SDK, and every nested invocation must re-enter the same exact dispatcher, policy checks, and journal as a direct call.

✕

The kid version: you carry one magic pen, but you must also carry an instruction card for every workshop tool the pen may use. First choose a small enough workshop shelf; then let the pen combine those tools without narrating every hammer swing.

DeepSeek Harness runs the incident

Assume an earlier admission step has already selected the handful of incident tools below their signatures fit in the SDK prompt:

MODEL  skill catalog (user-role reminder) → skill({name:"incident-response"})
       → <skill_content> tool result

MODEL  run_code(`
         const rollout = await k8s.getRolloutHistory({ns:"prod", deployment:"voice-ai"});
         const errors  = await datadog.queryLogs({query:"service:voice-ai status:error", since:"24h"});
         const limits  = errors.filter(e => e.status === 429).length;
         const diff    = await github.getPullDiff({repo:"voice-ai", number: rollout.prNumber});
         const verdict = limits > errors.length * 0.5 ? "provider rate limits" : "our deployment";
         const task    = await clickup.createTask({list:"incidents", name:`Voice AI failing: ${verdict}`, ...});
         await slack.postMessage({channel:"#ai-incidents", text: summarize(verdict, task.url)});
         return { verdict, taskUrl: task.url, errorCount: errors.length, rateLimited: limits };
       `)
       → { verdict: "provider rate limits", taskUrl: "…", errorCount: 4120, rateLimited: 3880 }

One top-level call composes six operations across four servers. Ordinary log records can stay inside the runtime and only a bounded return value needs to enter the conversation. But Code Mode did not find these operations: preselection did, and their SDK signatures were still in the prompt. A worker thread or language runtime is also not automatically a security sandbox; filesystem, network, process, time, memory, secret, and nested-call boundaries need independent enforcement.

MCP runs as one plugin instance per server, registering tools as mcp__<server>__<name>. On tools/list_changed it does a generational swap rather than accumulating, so stale tools cannot pile up.

Skills have the sharpest cache trick in the collection. The catalog is a sorted list of name plus capped description, around 500 characters max, omitting body, path, source, and whenToUse. It is injected as a durable user-role reminder message at pre-step, not as a system-prompt section. The catalog republishes only when its digest changes. Incomplete discovery keeps the last good catalog. If compaction hides the catalog message, the next complete snapshot re-injects it.

Why user-role? Because the system prefix is the most valuable cached bytes in the request. Adding or reordering a skill changes a message in the conversation, not the prefix. The tradeoff is authority: a user-role reminder carries slightly less instruction weight than system text.

Context has two seams. Compaction fires at about 80% of the window: an optional tool-result pruner, then an LLM summary that replaces a span with one user checkpoint, retaining about the most recent 16% verbatim. Separately, spill handles oversized current results: the model sees a preview plus a locator it can read or grep. Compaction rewrites history; spill keeps the present from blowing up.

The one difference that matters most

Strip away everything else and three shapes remain.

✕

The difference between rows one and two looks tiny. It is not.

Row one is easy for the model and hard for the provider. After search, the model’s next decision is one exact function call. But the provider has to understand its native typed activation item. Codex and Claude use different items, and unsupported providers need a bounded portable fallback—not the entire eager catalog.

Row two is easy for the provider and harder for the model. Any model that can call a function can call tool_search, tool_describe, and tool_call. But look at what the model must hold across generation boundaries after describe returns:

remember   tool name = mcp_crm_create_contact
remember   schema    = { firstName, locationId, email }
construct  tool_call(name="mcp_crm_create_contact", arguments={...})

That is a small protocol state machine. For a frontier model, trivial. For a cheap 20 to 30 billion parameter open model, it is three more places to make a mistake, and the study that produced these notes flags exactly this as Hermes’ cost. Codex and Claude Code collapse that state machine into “here is create_contact(firstName, locationId, email), call it”.

Row three is easy for the transcript and hard elsewhere. One call can suppress ordinary intermediate results, but the model still sees the selected SDK signatures, must write correct code, and needs a genuinely isolated runtime. A script that fails halfway also has no automatic rollback.

So the honest summary is:

Four bags, side by side

	Codex	Claude Code	Hermes Agent	DeepSeek Harness
Core tools	Direct exposure	Eager builtins	Normally eager, subject to defer overrides	Native, or replaced by run_code in code presentation
MCP at scale	Deferred + client BM25 tool_search	Prefiltered/partitioned request + defer_loading and ToolSearch	Session-scoped tool_search / describe / call bridge	Requires bounded preselection; Code Mode is presentation, not catalog search
Result of discovery	Typed tool_search_output carrying matched schema	Compact tool_reference	Tool name + metadata	No discovery primitive in the studied code path
Get the schema	Matched schema is in the typed load item	API expands the reference	Ordinary tool_describe result	Generated SDK section for selected tools
Execution	Exact namespaced tool	Exact native tool	tool_call(name, args), harness unwraps	Script calls selected tools through the runtime
Provider dependence	High	High	Low	Low at protocol level; needs certified isolation
Visibility guard	Bounded namespace guidance	Bounded routing guidance for admitted candidates	Degradation ladder under min(5%, 4k tokens), possibly no flat listing	SDK lists all selected signatures and remains O(N)
Skill index	Name + desc + locator, 2% budget, round-robin trim	Name + desc, 1% budget, trim least-used first	60-char desc in volatile tier	Sorted name + ~500-char desc, user-role message
Skill body	Read SKILL.md, 8 KB cap	Injected as one message	skill_view tool result	skill() tool result
Big results	Truncate with size note	Spill to disk above 25k tokens	Per-tool result budgets	Spill: preview + locator
After compaction	Re-inject initial context + catalog	Re-inject CLAUDE.md, memory, 5 files, skills ≤25k	[SKILL_PRUNED] → reload	Next snapshot re-injects catalog
Model sees 50k schemas?	Not initially; matched schemas enter typed history	No, and one request admits at most 10k deferred definitions	No	It must not: selection has to happen before SDK generation
Known gap	Native path needs compatible provider; matched-schema history cost	Candidate schemas still cross the wire; 10k ceiling	Generic executor taxes small models; extreme-scale recall unproven	SDK is O(N); isolation and rollback are external responsibilities

Notice the shared direction: the registered universe should not become the model’s initial schema view; procedural skill bodies should load on demand; results need bounds; and compaction cannot be the authority for capabilities. The implementations do not agree on wire cost, typed history, preselection, or whether discovery exists at all.

They disagree on how to finish: schema-bearing typed load, compact reference, generic call, or code over an already-selected SDK.

Discovery happens at four levels, and the model should see one

Go back to our monster company and one line of the task: “create a ClickUp incident”. Hidden inside that are at least six decisions.

which skill?          incident-response, maybe a clickup-ops skill
which source?         ClickUp MCP, not the ClickUp plugin's native tool
which operation?      create_task, not create_list or create_folder
which account?        the engineering workspace, the incidents list
which schema?         exactly this revision of create_task
are we authorized?    scopes, tenant, approval policy
then execute

The bad shape makes the model walk all six, every time:

MODEL: choose plugin → choose connection → choose skill → load guide
       → choose action → remember action → build arguments → use_plugin(...)

That is the model doing the harness’s bookkeeping. Plugin slugs, connection ids, skill ids, and schema revisions have become model-visible plumbing, and a small model will fumble one of them.

The good shape resolves everything it can outside the model and shows the model one thing:

HARNESS resolves:  plugin · connection · authorization · catalog · operation identity · schema revision

MODEL sees:        create_task({ list, name, description })

Plugins sit one level above skills and tools, and they should stay there. Installing an “Engineering Operations” plugin puts its three skills into the skill catalog, its two MCP servers into the deferred tool catalog, and its commands into the command menu. The model never needs a “first pick a plugin” step. Flatten the executable universe; federate the search over it.

What none of them has solved

The studied mechanisms handle tool schemas more deliberately than they handle very large skill catalogs. Skill metadata is bounded and eager rather than production-routed through search. Codex can trim descriptions and eventually drop tail entries; Claude Code can strip descriptions until only names remain; Hermes and DeepSeek also cap their advertisements. Our hypothetical 250 skills may fit, but at 2,500 a name-only entry is a weak routing signal.

The honest extension, which none of these ships in production yet, is a searched skill index that still keeps a tiny category-and-name listing visible, so the search itself is discoverable. Codex has the BM25 selectors in shadow mode. That is the closest anyone is.

Search does not make the model choose correctly. It only makes choosing possible. Large-catalog retrieval therefore needs explicit recall, abstention, first-call validity, and end-to-end success evaluation for each exact model and provider build.

If you are building one

Pack the bag like this. It is a hybrid, and the study these notes come from lands on the same one.

Keep from Hermes: bounded BM25 search over the session-scoped toolset, deterministic advertisement degradation, and provider neutrality. Hermes performs that search in the harness process; moving it to a shared server-side federated index is an architectural adaptation for a distributed system, not observed Hermes behavior.

Keep from Codex and Claude Code: exact schemas, typed activation, and native execution of the found tool. Retained provider state can avoid repeated discovery, but persist canonical capability identity and revisions separately so compaction or failover can reconstruct it.

Keep from DeepSeek: the skill catalog as a conversation message rather than a prefix mutation, spill for oversized results, and Code Mode for a bounded selected set when the model, isolation profile, policy, cancellation, and evidence handling have been certified.

Then the checklist.

1. Keep a narrow core. Tens of tools, direct, paid every turn, and be stingy about who gets in. Try a CLI plus a skill before a new tool.
2. Defer the rest behind bounded search. Keep the search control visible and advertise categories or names within a hard budget; do not pretend every operation can stay individually visible forever.
3. Search only when necessary, then activate the exact operation. The model should end up calling create_task(...), not tool_call("clickup_create_task", ...).
4. Use typed activation without mutating the top-level tool prefix. Do not dump JSON Schema as prose. Be explicit about remaining costs: Codex can carry the matched schema in structured history, while Claude carries admitted candidate schemas in the API tools collection.
5. Skills are documents. Index them, load them into a tool result, re-attach the loaded ones after compaction.
6. Spill big results to disk. Preview plus locator. The transcript is an attention budget, not an archive.
7. Restore, do not summarize, the durable layers. Repo instructions, skill catalog, environment state.
8. Resolve plumbing outside the model. Plugins, connections, accounts, schema revisions, authorization. Show the model one function.
9. Treat discovery as discovery, not permission. Recheck live authorization, connection readiness, schema compatibility, approval, and idempotency at dispatch.

The kid version of all of it: carry the card catalog, not the library. Know what you own. Fetch the book when you need it. Read the exact page, not a note about which page. Put it back when you are done.

I’m Kaushal Prajapati, a Staff AI/ML Engineer. Explore more posts, or say hi on LinkedIn.