Screaming Frog Mcp

Abstract

1. Introduction

A headless MCP (Model Context Protocol) server for Screaming Frog SEO Spider. It drives the SF command line and saved crawl database directly, so Claude (or any MCP-compatible client) can run crawls, export crawl data, and analyze the results with the Screaming Frog GUI closed: on your laptop, on a server, or inside scheduled audits and CI pipelines.

This is a community project, not affiliated with Screaming Frog. Since SEO Spider v24 there is also an official MCP built into the app. The two work differently and solve different problems.

Screaming Frog shipped an official MCP server in SEO Spider v24. It's substantial: around 29 tools covering crawl control (start, pause, resume, progress), reports and bulk exports with field selection, URL-level inspection, screenshots, embeddings exports, and optionally a Node.js script runner with npm and filesystem read/write tools. It runs in two modes, either a Streamable HTTP server inside the open app, or a STDIO mode where the MCP client launches the Spider itself, headless. Setup is documented for Claude Desktop and LM Studio.

2. Objective

Headless MCP server for Screaming Frog SEO Spider – run crawls, export and analyze crawl data via the CLI. Small locked-down tool surface for scheduled audits, CI, and unattended AI agents.

This project demonstrates how Python can be applied to a real-world Digital Marketing & SEO problem.

4. Technology Stack

5. System Requirements

General requirements for this technology stack — check the README for exact versions.

• Python 3.8 or later
• pip / virtualenv for dependencies
• VS Code, PyCharm or Jupyter Notebook
• Git (to clone the repository)

6. Installation & Setup

git clone https://github.com/bzsasson/screaming-frog-mcp.git
cd screaming-frog-mcp

uv tool install screaming-frog-mcp

pip install screaming-frog-mcp

git clone https://github.com/bzsasson/screaming-frog-mcp.git
cd screaming-frog-mcp
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Full setup instructions are in the project README.

7. Future Enhancements

Suggested extensions you can add to make this your own project.

• Export reports to Google Sheets or PDF
• Schedule weekly automated reports
• Add competitor comparison

8. Viva / Review Questions

Common questions examiners ask for projects in this domain.

1. Which marketing or SEO problem does this tool solve?
2. Which data sources or APIs does it use (Search Console, Analytics, social platforms)?
3. Which metrics or KPIs does it report and how are they calculated?
4. How could the output help a business make decisions?
5. How would you schedule it to run automatically?

9. Source Code & License

This project is developed by bzsasson and published on GitHub under the MIT License. Please follow the license terms and credit the original author when you use or modify this code.