Webcrawl MCP server

Webcrawl is a small helper that lets your AI assistant actually open web pages, search the internet, and walk through a website for you. It runs on your own computer and uses free, open tools to pull the text out of pages. It is handy if you often paste links into your AI and wish it could just read them itself.

What this MCP server does

You ask your AI something like "read this page and summarize it" or "search for X and tell me what you find". Your AI passes that request to Webcrawl, which fetches the page or runs a DuckDuckGo search on your behalf. Webcrawl strips out the clutter and hands back the main text of the page as clean markdown. Your AI then reads that text and answers you in the chat. Each result also carries a small note saying where the text came from, so you can see whether it was fetched normally or through the optional Firecrawl backup.

Click to zoom

What you can do with it

• Fetch a single web page and get its main text back
• Search the web with DuckDuckGo and optionally read the top results
• List the links found on a page so you can see what else is on the site
• Crawl several pages of a site in one go and collect their text
• Summarize an article or documentation page without leaving the chat
• Gather text from a few pages to compare or research a topic

Try asking your AI

• “Read this page and give me a short summary: https://docs.python.org/3/library/asyncio.html”
• “Search for the latest news about electric cars and tell me the main points”
• “List all the links on example.com so I can see what sections the site has”
• “Crawl the first few pages of this blog and pull out the titles and key ideas”

What it gives back to you

You get the main text of each page back as clean markdown, ready for your AI to read and summarize. Search results come back as a list of titles, links, and short snippets, and if you ask it to read them, the page text is included too. Crawl and map results come back as a list of pages with their text or links. Everything appears right in your chat, so you can keep asking follow-up questions.

Before you start

Who it's for

Anyone who reads a lot of web pages for work, like researchers, writers, analysts, or developers who want their AI to fetch and summarize pages for them.