Agent-Wiz: Analyzing AI Intelligentsia Workflows and Security Risks

General Introduction

Agent-Wiz is an open source Python command line tool designed for developers, researchers and security teams. It extracts complex workflows from mainstream AI intelligences frameworks such as LangGraph, CrewAI, AutoGen, etc., generates intuitive visual diagrams, and automatically analyzes potential security risks with the MAESTRO Threat Modeling Framework.Agent-Wiz provides users with a clear understanding of the logic of the interactions between intelligences, tools, and data streams, and is suitable for debugging code, optimizing design, or assessing system security.

Function List

• Extract the workflow of AI intelligences to generate detailed node and connection data.
• Generate interactive visualization charts that show the dynamic relationships between intelligences, tools, and data flows.
• Threat modeling based on MAESTRO framework and output security risk reports.
• Supports a variety of AI frameworks, including AutoGen, CrewAI, LangGraph, LlamaIndex, and more.
• Provides a simple command-line interface , supports JSON format output , easy to integrate developers .
• Allow users to customize the parser and extend support for new frameworks.

Using Help

Agent-Wiz is a lightweight command line tool, based on Python, easy to install and powerful. Below is a detailed installation and usage guide to help users get started quickly and take full advantage of its features.

Installation process

Agent-Wiz requires Python 3.8 or higher and relies on the OpenAI API for some of its functions. Here are the installation steps:

1. Installing Agent-Wiz
   Open a terminal and run the following command to install directly via pip:

   pip install repello-agent-wiz
   

   This will automatically install Agent-Wiz and its dependent libraries, such as Click, Graphviz, and so on.

2. Setting the OpenAI API Key
   Some features (such as threat modeling) require OpenAI API support. Run the following command to set environment variables:
   • macOS/Linux:

     export OPENAI_API_KEY=sk-...
     

   • Windows:

     set OPENAI_API_KEY=sk-...
     

   To persist the settings, you can add the above command to the.bashrc,.zshrcor in an environment variable configuration file.

3. Verify Installation
   After the installation is complete, run the following command to check for success:

   agent-wiz --help
   

   If a command help message is displayed, the installation was done correctly.

Usage

Agent-Wiz provides three core commands:extract(Extraction workflow),visualize(generating visualization charts) andanalyze(Perform threat modeling). The following are detailed instructions for doing so.

1. Extraction workflow

This function extracts the workflow of an intelligent body by statically parsing the code. Assuming you have a CrewAI-based project, run the following command:

agent-wiz extract --framework crewai --directory ./my_project --output workflow.json

• --framework: Specify the framework to supportautogen,crewai,langgraph,llamaindex,n8n,swarmetc.
• --directory: Specify the project folder path, Agent-Wiz will analyze the code structure.
• --output: Specify the output file path to generate workflow data in JSON format.
  The output JSON file contains nodes (intelligences, tools, etc.) and edges (interactions) with the following structure:

{
"nodes": [{"id": "agent1", "type": "agent"}, ...],
"edges": [{"from": "agent1", "to": "tool1"}, ...],
"metadata": {"framework": "crewai"}
}

2. Generation of visualization charts

The visualization feature converts the workflow into an interactive diagram that shows the connections between intelligences. Run the following command:

agent-wiz visualize --input workflow.json --open

• --input: Specifies the JSON file to be extracted.
• --open: Optional, automatically opens the generated HTML chart in the default browser.
  Charts are based on D3.js and show dynamic relationships between intelligences, tools and data flows, with users zooming and dragging to view details.

3. Conducting threat modeling

The Threat Modeling feature is based on the MAESTRO framework and analyzes security risks in the workflow. Run the following command:

agent-wiz analyze --input workflow.json

• --input: Use the extracted JSON file.
  After running, the tool generates a report in Markdown format (e.g.crewai_report.md), content included:
• List of system objectives and assets.
• Attack portals and potential threats.
• Risk assessment and safety recommendations.
  The MAESTRO framework specifically analyzes the following aspects:
• mission objective: Clarify system functionality and security requirements.
• liabilities: List the intelligences, tools and data streams.
• entrances: Identify possible points of attack.
• control: Evaluate existing defense mechanisms.
• menace: List vulnerabilities and attack scenarios.
• exposures: Calculate impacts and likelihoods.
• scheduled service (train, bus etc): Check for operational safety issues.

4. Viewing help information

For more command details, run:

agent-wiz --help

This will list all available commands and their parameters to help the user explore the functionality.

Supported Frameworks

Agent-Wiz supports the following mainstream AI intelligences frameworks, each with a specialized AST parser to ensure accurate extraction:

• AutoGen: Support for core functionality and AgentChat modules.
• CrewAI: Parsing Teamwork Intelligence.
• LangGraph: Handling complex directed graph workflows.
• LlamaIndex: Support for data indexing and query intelligences.
• n8n: Parsing automated workflows.
• OpenAI Agents: Compatible with OpenAI's Intelligent Body API.
• Pydantic-AI: Support for Pydantic-based intelligences.
• Swarm: Handling multi-intelligence collaboration.

Advanced Use

• batch analysis: Write scripts to loop throughextractcap (a poem)visualize, handling multiple projects.
• Customized Outputs: Adjust the chart style and refer to the D3.js configuration in the project documentation.
• Extended Framework: Developers can write new parsers based on the SDK to extend support for other frameworks.
• Security report optimization: Refine threat modeling reports in conjunction with manual review.

caveat

• Ensure that the project code contains a clear structure and comments for easy parsing.
• The OpenAI API key needs to be valid or the threat modeling feature is not available.
• If you encounter dependency issues, run the following command to upgrade pip:

  pip install --upgrade pip
  

• The visualization charts are recommended to be viewed in modern browsers such as Chrome for the best experience.

With these steps, users can easily install Agent-Wiz to extract workflows, generate charts and analyze security risks.

application scenario

1. Developers Optimize Intelligent Body Systems
   When developing LangGraph intelligences, developers may have difficulty making sense of complex interaction logic. Agent-Wiz extracts the workflow and generates diagrams to help quickly pinpoint problems and improve system efficiency.
2. Security team identifies potential threats
   Security experts need to assess AutoGen intelligences for vulnerabilities. the MAESTRO threat modeling feature of Agent-Wiz generates a detailed report that lists attack entry points and recommendations for improvement.
3. Technical training to demonstrate the AI framework
   The training instructor uses Agent-Wiz to generate interactive diagrams of CrewAI workflows to visualize the principles of multi-intelligence collaboration to students and lower the learning threshold.
4. Researchers analyze new framework
   When researchers explore emerging frameworks such as Swarm, they can use Agent-Wiz to extract their workflows, compare the advantages and disadvantages of different frameworks, and assist academic research.

QA

1. Does Agent-Wiz require an OpenAI API?
   The threat modeling function requires an OpenAI API key, and the extraction and visualization functions do not require an API.
2. What threat modeling frameworks are supported?
   MAESTRO is currently supported, and frameworks such as STRIDE and PASTA are under development.
3. Can you handle large projects?
   Yes, but it is recommended that the project code is well structured and parsed more efficiently.
4. Can the output charts be exported to other formats?
   HTML and JSON are currently supported, and PNG or SVG may be supported in the future.