WebFetchTool: Fetch webpages

WebFetchTool is responsible for fetching a web page whose URL is already known, converting the page content into text that is easier for Claude Code to process, and then extracting results based on the provided prompt. Its division of labor with WebSearchTool is very clear:

• WebSearchTool: Helps the model find results on the web
• WebFetchTool: Takes a confirmed URL to read a specific page So you can understand it as:

A "web page reader + small model summarizer" for Claude Code rather than an "internet search tool."

tools/WebFetchTool/WebFetchTool.ts：

const inputSchema = z.strictObject({
  url: z.string().url().describe('The URL to fetch content from'),
  prompt: z.string().describe('The prompt to run on the fetched content'),
})

These two fields exactly correspond to its two-step workflow:

1. Fetch the web page content first
2. Then extract the desired information based on prompt Therefore, WebFetchTool is not merely "downloading a web page," but combines "fetching" and "reading" into a single complete action.

Model already knows target URL > WebFetchTool > URL validation > Domain permission check > Fetch web page > HTML to Markdown / Handle binary content > Pass markdown and prompt to small model > Return refined result

This is the most notable design point of WebFetchTool. In tools/WebFetchTool/utils.ts and prompt.ts, you can see that it passes the fetched web page content to a secondary model for processing:

import { queryHaiku } from '../../services/api/claude.js'
import { makeSecondaryModelPrompt } from './prompt.js'

And in prompt.ts, the prompt assembly for this step is defined as:

export function makeSecondaryModelPrompt(
  markdownContent: string,
  prompt: string,
  isPreapprovedDomain: boolean,
): string {
  return `
Web page content:
---
${markdownContent}
---

${prompt}
`
}

This indicates that its actual workflow is:

1. Fetch web page
2. Convert to markdown
3. Pass the markdown and the user/model's question together to a smaller, faster model
4. Return only the refined result to the main loop Compared to "stuffing the entire HTML page into the main context," this saves a significant amount of context.

WebFetchTool is not a browser.
It acts more like a remote reader focused on content extraction. It is not good at:

• Interacting with authenticated/login-state pages
• Clicking, forms, dynamic operations
• Simulating complex JS page behaviors And what it excels at are:
• Documentation pages
• Blog posts
• Public-facing documentation pages
• Structured summaries of specific URLs Because of this, Anthropic also includes a specific reminder in the prompt:

IMPORTANT: WebFetch WILL FAIL for authenticated or private URLs.
If so, look for a specialized MCP tool that provides authenticated access.

If this is an authenticated page, private document, or permission-protected site, WebFetchTool will likely fail.
In that case, you should look for an MCP tool or go through another authenticated mechanism.

WebFetchTool contains a crucial permission logic:

function webFetchToolInputToPermissionRuleContent(input) {
  const { url } = parsedInput.data
  const hostname = new URL(url).hostname
  return `domain:${hostname}`
}

The subsequent checkPermissions() will evaluate based on this domain:hostname:

• deny
• ask
• allow This indicates that Claude Code does not simply say "allow internet access/disable internet access," but further refines web fetch permissions to the domain level.

Input URL > Extract hostname > Generate domain:xxx permission key > Check deny / ask / allow rules > Allow fetch or request user confirmation This design closely resembles a truly productized security model.

There are two very important functions in the source code:

import { isPreapprovedHost } from './preapproved.js'
import { isPreapprovedUrl } from './utils.js'

This means that certain domains recognized by the system will follow a smoother fetch path.
And in makeSecondaryModelPrompt(), different summarization constraints are generated depending on whether it is a pre-approved domain. For ordinary domains, it additionally emphasizes:

• Quotes must be short
• Do not extensively paraphrase the original text
• Do not cross into discussing legal issues
• Do not generate sensitive long quotes such as lyrics This is essentially managing copyright and content risk.

In utils.ts, you can see:

const CACHE_TTL_MS = 15 * 60 * 1000
const MAX_CACHE_SIZE_BYTES = 50 * 1024 * 1024

const URL_CACHE = new LRUCache<string, CacheEntry>({
  maxSize: MAX_CACHE_SIZE_BYTES,
  ttl: CACHE_TTL_MS,
})

This indicates that WebFetchTool caches the content of recently fetched webpages.
The benefits of this are:

• Faster repeated fetches for the same URL
• Reduced network overhead
• Less redundant model processing

It even maintains a separate pre-check cache for domains:

const DOMAIN_CHECK_CACHE = new LRUCache<string, true>({
  max: 128,
  ttl: 5 * 60 * 5000,
})

These kinds of details are highly characteristic of a mature product, rather than a demo tool.

Inside validateURL(), the code doesn't just "allow it to parse," it also enforces limits on:

• URL length
• Whether it contains a username or password
• Whether it appears to be a public domain

The source code also reveals this line:

const MAX_URL_LENGTH = 2000

as well as preemptive filtering for internal, anomalous, or unsuitable targets.
This shows Anthropic understands very well: web fetching is a capability that can easily become a security vector, so guardrails are necessary.

WebFetchTool does not blindly follow cross-origin redirects.
In the source code, it verifies whether a redirect is safe:

export function isPermittedRedirect(
  originalUrl: string,
  redirectUrl: string,
): boolean

Roughly permitted are:

• Navigations within the same domain
• Toggling the www. prefix
• Path adjustments

However, if it redirects to a different host, it does not silently continue fetching; instead, it returns a definitive response, leaving the decision to the main thread. Within WebFetchTool.ts, you'll find this highly characteristic logic:

if ('type' in response && response.type === 'redirect') {
  const message = `REDIRECT DETECTED: The URL redirects to a different host. ...`
}

This demonstrates that "cross-origin redirects" are treated as security events requiring explicit handling, rather than as standard network behavior.

The utils.ts file includes:

import {
  isBinaryContentType,
  persistBinaryContent,
} from '../../utils/mcpOutputStorage.js'

This indicates that the content fetched by WebFetchTool isn't always HTML text.
For binary responses, it routes through persistence handling rather than forcing the content into a text format. This further confirms that its implementation goal isn't to "download just anything," but to "securely process web content within strict product constraints."

A typical workflow proceeds as follows:

1. The main thread uses WebSearchTool to search for a topic
2. Selects one of the result URLs
3. Calls WebFetchTool
4. Uses a prompt to specify "extract exactly what I care about"
5. Returns the summarized results to the main thread

This distinction is crucial to understand:

• WebSearchTool is responsible for "locating potential sources"
• WebFetchTool is responsible for "reading the actual pages"

You can think of them this way:

WebSearchTool > Finds candidate URLs > WebFetchTool > Reads and extracts page content

In many web-assisted tasks, both tools are used in tandem.

That's incorrect.
It also performs:

• Access validation
• Domain verification
• HTML-to-Markdown conversion
• Lightweight model-based summarization
• Response caching
• Secure redirect handling

This is also inaccurate.
It is better suited for static or directly accessible content pages, rather than complex authenticated sessions or interactive UIs.

They operate on completely different levels.
One handles discovery; the other handles consumption.

If summarized in one sentence:

WebFetchTool acts as Claude Code’s "targeted web fetcher," consolidating URL resolution, content sanitization, lightweight model summarization, domain validation, and security controls into a formal read-only utility.

The real strength of this tool lies not in its ability to connect to the web, but in:

it manages to execute safely by keeping risks, context, and output scale strictly under control.