Agent Audit

Abstract

1. Introduction

AI agents are not just chatbots. They execute code, call tools, and touch real systems, so one unsafe input path can become a production incident.

Think of it as security linting for AI agents, with 66 rules mapped to the OWASP Agentic Top 10 (2026).

--severity controls what is reported. --fail-on controls when the command exits with code 1.

2. Objective

Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 51 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.

This project demonstrates how Python can be applied to a real-world Cybersecurity & Ethical Hacking problem.

4. Technology Stack

5. System Requirements

General requirements for this technology stack — check the README for exact versions.

• Python 3.8 or later
• pip / virtualenv for dependencies
• VS Code, PyCharm or Jupyter Notebook
• Git (to clone the repository)

6. Installation & Setup

git clone https://github.com/HeadyZhang/agent-audit.git
cd agent-audit

1. Scan your project
2. Interpret and gate in CI
3. Precision 68.86%, Recall 84.32%, F1 0.7581 (raw, reproducible) — TP 199 / FP 90 / FN 37
4. OWASP Agentic Top 10 coverage: 10/10

pip install agent-audit

agent-audit scan ./your-agent-project

# Show only high+ findings
agent-audit scan . --severity high

# Fail CI when high+ findings exist
agent-audit scan . --fail-on high

# Scan a project
agent-audit scan ./my-agent

# JSON output for scripting
agent-audit scan ./my-agent --format json

# SARIF output for GitHub Code Scanning
agent-audit scan . --format sarif --output results.sarif

# Only fail CI on critical findings
agent-audit scan . --fail-on critical

# Inspect a live MCP server (read-only, never calls tools)
agent-audit inspect stdio -- npx -y @modelcontextprotocol/server-filesystem /tmp

Full setup instructions are in the project README.

7. Future Enhancements

Suggested extensions you can add to make this your own project.

• Add logging and alert notifications (email / Telegram)
• Write a threat model document for the tool
• Package it with Docker for safe lab testing

8. Viva / Review Questions

Common questions examiners ask for projects in this domain.

1. Which threat or attack does this project defend against?
2. What detection or protection technique is used and what are its limits?
3. How are false positives and false negatives handled?
4. Which cryptographic algorithms or security standards are involved?
5. What legal and ethical rules apply when testing a tool like this?

9. Source Code & License

This project is developed by HeadyZhang and published on GitHub under the MIT License. Please follow the license terms and credit the original author when you use or modify this code.