Documentation Scraper MCP server

This is a small helper program that copies documentation websites onto your own computer as clean text files. It is handy if you often ask your AI about a tool or a library and want it to answer from the real docs instead of guessing. It runs on your machine, so the files stay with you.

What this MCP server does

You point the tool at a documentation website and it walks through the pages, one by one, and saves each one as a clean text file on your computer. It keeps the same folder structure as the original site, so things stay easy to find. When you ask your AI a question, the AI uses this helper to search those saved files and pull out the parts that match. You then get an answer based on the real documentation, plus the pages it read. It can also tell you when the saved copy was last refreshed and what changed since the previous crawl.

Click to zoom

What you can do with it

• Save a whole documentation site to your computer as readable text files
• Search your saved docs offline and get ranked results
• Read a specific documentation page and get its content back
• Check how fresh your saved copy is
• Compare two crawls and see what pages were added, changed, or removed
• Re-crawl a site on a schedule so your copy stays up to date
• Resume a crawl that was interrupted instead of starting over

Try asking your AI

• “Search my saved PyTorch docs for how to freeze layers in a model”
• “Read the page about installation from the docs I crawled yesterday”
• “Is my copy of the Rust CLI book still fresh, or should I re-crawl it?”
• “What changed in the TensorFlow docs since the last crawl?”

What it gives back to you

You get answers in the chat, usually with short quotes or snippets from the pages that matched. For a page read, you get the cleaned text of that page. For a freshness check, you get a simple status and dates. For a diff, you get a list of pages that were added, changed, or removed.

Before you start

Who it's for

People who work with technical documentation a lot, like developers, technical writers, support engineers, and anyone who wants their AI to answer from real docs instead of memory.