We release patches for security vulnerabilities in the following versions:

We take the security of AgentUnit seriously. If you believe you have found a security vulnerability, please report it to us as described below.

• Open a public GitHub issue for security vulnerabilities
• Disclose the vulnerability publicly before it has been addressed

For security vulnerabilities, please email: opensource@agentunit.dev

In your report, please include:

1. Description - Clear description of the vulnerability
2. Impact - What an attacker could do with this vulnerability
3. Steps to Reproduce - Detailed steps to reproduce the issue
4. Affected Versions - Which versions are affected
5. Suggested Fix - If you have ideas on how to fix it
6. Your Contact Info - How we can reach you for follow-up

1. Acknowledgment - We will acknowledge receipt of your report within 48 hours
2. Initial Assessment - We will provide an initial assessment within 5 business days
3. Regular Updates - We will keep you informed of our progress
4. Fix Timeline - Critical vulnerabilities will be addressed within 7-14 days
5. Public Disclosure - We will coordinate disclosure timing with you
6. Credit - You will be credited in the security advisory (unless you prefer to remain anonymous)

When a security vulnerability is confirmed:

1. We will develop and test a fix
2. We will prepare a security advisory
3. We will release a patch version
4. We will publish the security advisory
5. We will notify users through:
   • GitHub Security Advisories
   • Release notes
   • PyPI release announcement

• Always install from official sources (PyPI)
• Verify package integrity
• Keep AgentUnit updated to the latest version

# Check current version
pip show agentunit

# Update to latest version
pip install --upgrade agentunit

• Never commit API keys or secrets to version control
• Use environment variables for sensitive configuration
• Rotate API keys regularly
• Use least-privilege principles for API access

Example secure configuration:

import os
from agentunit import Scenario

# Good: Use environment variables
api_key = os.environ.get("OPENAI_API_KEY")

# Bad: Never hardcode secrets
# api_key = "sk-..."

• Sanitize sensitive data before evaluation
• Use AgentUnit's privacy features for PII masking
• Review trace logs before sharing
• Be aware of data retention in external services

• Secure your OTLP endpoint with TLS
• Use authentication for telemetry backends
• Filter sensitive attributes from traces
• Review exported data regularly

AgentUnit integrates with various third-party services (OpenAI, Anthropic, etc.):

• Each adapter may have its own security considerations
• Review third-party service security policies
• Understand data processing and retention policies
• Use adapter-specific security features

• Test datasets may contain sensitive information
• Be cautious when sharing evaluation results
• Use mock data for public demonstrations
• Review metric outputs for information leakage

• AgentUnit may execute adapter code dynamically
• Only use trusted adapters from verified sources
• Review custom adapter code before deployment
• Use sandboxing for untrusted scenarios

We monitor dependencies for known vulnerabilities using:

• GitHub Dependabot
• Safety checks in CI
• Regular dependency audits

To check for vulnerable dependencies:

# Using safety
pip install safety
safety check

# Using pip-audit
pip install pip-audit
pip-audit

AgentUnit supports privacy-preserving evaluation:

• PII detection and masking
• Differential privacy mechanisms
• Federated evaluation
• Secure aggregation

See Privacy Documentation for details.

• AgentUnit is MIT licensed
• Review licenses of all dependencies
• Ensure compliance with third-party service terms
• Respect data usage restrictions

• Added comprehensive security documentation
• Improved PII detection capabilities
• Enhanced telemetry filtering options

• Initial security policy
• Basic PII masking features

• Security Email: opensource@agentunit.dev
• General Issues: GitHub Issues
• Discussions: GitHub Discussions

We appreciate responsible disclosure and will acknowledge security researchers who help us maintain a secure project.

Thank you for helping keep AgentUnit secure!