WhatsUpDoc: Docs to Markdown & MCP Comms

什么是WhatsUpDoc

WhatsUpDoc（downmarked）是一个轻量级命令行工具，专注于从网页快速抓取技术文档并转换为Markdown格式。通过集成MCP通信协议，它能高效解析网页结构，支持本地化存储，特别适合开发者需要离线查阅文档的场景。其核心优势在于自动化处理复杂网页，同时保留代码块、表格等关键内容。

如何使用

安装过程简单：只需通过npm全局安装

npm install -g downmarked

基础用法只需一条命令即可完成文档抓取与转换：

downmarked https://example.com/docs -o output.md

通过参数可灵活控制行为：指定输出路径（-o）、设置CSS选择器过滤内容（--selector）、启用递归抓取子页面（--recursive）等高级选项。

关键特性

• 智能内容识别：自动检测并保留代码块、表格、标题等结构化内容
• 深度爬取能力：通过--recursive参数可构建完整的文档知识库
• 精准内容裁剪：使用CSS选择器精确控制需要抓取的页面区域
• 格式完美转换：生成的Markdown格式与原文档视觉结构高度一致
• 跨平台支持：基于Node.js开发，兼容Windows/macOS/Linux系统

使用案例

前端开发者常用于：

1. 将React官方文档离线化，方便地铁通勤时查阅
2. 将Python官方教程转换为便于版本控制的Markdown格式
3. 构建技术文档的自动化归档系统，定期更新云存储中的文档

常见问题

需要安装额外依赖吗？

仅需Node.js运行环境，其他依赖已打包在工具中

能处理动态加载的网页吗？

当前版本主要支持静态网页，复杂JavaScript渲染内容可通过浏览器扩展辅助

如何贡献改进？

请访问项目主页 GitHub仓库 提交Issue或Pull Request

WhatsUpDoc (downmarked)

A command-line tool for fetching and storing developer documentation locally using the Model Context Protocol (MCP).

Features

• Fetch documentation from any website and convert it to Markdown
• Save documentation to any location on your system
• Target specific content using CSS selectors
• Recursively fetch linked documentation pages
• Split documentation by headers into separate files
• Uses the Model Context Protocol (MCP) for standardized communication

Installation

# Install globally
npm install -g downmarked

# Or use with npx
npx downmarked fetch https://reactjs.org/docs/getting-started.html

Usage

Basic Usage

downmarked fetch <url>

This will prompt you for an output location and save the documentation as Markdown.

Options

# Fetch documentation with specific options
downmarked fetch https://reactjs.org/docs/getting-started.html \
  -o ~/Documents/react-docs.md \
  -s "main" \
  -r \
  -d 2 \
  --split

Available Options

Option	Description
-o, --output <path>	Output path (absolute or relative)
-s, --selector <selector>	CSS selector to target specific content
-r, --recursive	Recursively fetch linked documentation pages
-d, --max-depth <number>	Maximum depth for recursive fetching (default: 3)
--split	Split documentation by headers into separate files

Examples

Fetch React Documentation

# Save React documentation to a specific location
downmarked fetch https://reactjs.org/docs/getting-started.html -o ~/Documents/react-docs.md

# Target only the main content area
downmarked fetch https://reactjs.org/docs/getting-started.html -s "main"

# Recursively fetch linked pages up to 2 levels deep
downmarked fetch https://reactjs.org/docs/getting-started.html -r -d 2

Fetch Python Documentation

# Save Python documentation
downmarked fetch https://docs.python.org/3/tutorial/index.html -o python-tutorial.md

How It Works

WhatsUpDoc (downmarked) uses the Model Context Protocol (MCP) to standardize communication between the CLI and the documentation server. The tool:

1. Fetches HTML content from the specified URL
2. Parses the HTML using Cheerio
3. Converts the HTML to Markdown using Turndown
4. Saves the Markdown to the specified location

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Model Context Protocol (MCP) for providing the communication framework
• Turndown for HTML to Markdown conversion
• Cheerio for HTML parsing