Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,15 @@ poetry run python3 -m pytest tests/integration/ -v
Latest verification (2025-10-24): 144 passed, 10 skipped, 32 warnings. Warnings originate from third-party dependencies (`langchain` pydantic shim deprecations and `datetime.utcnow` usage). Track upstream fixes or pin patched releases as needed.

### Running CI Checks Locally
#### Usage

Run all checks (same as CI):

```bash
poetry run ruff check .
poetry run ruff format --check .
poetry run pytest tests -v
```

Before opening a pull request, you can run the same checks locally that are executed in CI.

Expand Down
8 changes: 8 additions & 0 deletions report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# AgentUnit Report

## emoji-suite
Success rate: 50.00%

- **test_pass**: ✅
- **test_fail**: ❌
- Error: AssertionError
4 changes: 2 additions & 2 deletions src/agentunit/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""

from __future__ import annotations

from .core.runner import Runner, run_suite
Expand All @@ -8,6 +6,8 @@
from .reporting.results import ScenarioResult, SuiteResult


"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""

Comment on lines +9 to +10
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Move the module docstring to the correct position.

Module docstrings must be the first statement in a file (immediately after from __future__ imports). When placed after regular imports, the string becomes a discarded literal and won't be accessible via help(agentunit) or agentunit.__doc__.

🔎 Proposed fix
 from __future__ import annotations
 
+"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""
+
 from .core.runner import Runner, run_suite
 from .core.scenario import Scenario
 from .datasets.base import DatasetCase, DatasetSource
 from .reporting.results import ScenarioResult, SuiteResult
 
 
-"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""
-
 __all__ = [
     "DatasetCase",
     "DatasetSource",
🤖 Prompt for AI Agents
In src/agentunit/__init__.py around lines 9-10, the module docstring is placed
after regular imports so it becomes a discarded literal; move the triple-quoted
docstring to be the very first statement in the file (immediately after any from
__future__ imports if present) so it becomes the module-level docstring
accessible via help() and __doc__.

__all__ = [
"DatasetCase",
"DatasetSource",
Expand Down
51 changes: 51 additions & 0 deletions tests/test_reporting.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
from datetime import datetime
from pathlib import Path

from agentunit.reporting.results import (
ScenarioResult,
ScenarioRun,
SuiteResult,
)


def test_markdown_contains_emojis():
passing_run = ScenarioRun(
scenario_name="emoji-suite",
case_id="test_pass",
success=True,
metrics={},
duration_ms=5,
trace=[],
error=None,
)

failing_run = ScenarioRun(
scenario_name="emoji-suite",
case_id="test_fail",
success=False,
metrics={},
duration_ms=6,
trace=[],
error="AssertionError",
)

scenario = ScenarioResult(
name="emoji-suite",
runs=[passing_run, failing_run],
)

suite = SuiteResult(
scenarios=[scenario],
started_at=datetime.now(),
finished_at=datetime.now(),
)

output_path = Path("report.md")
suite.to_markdown(output_path)
# UTF-8 safety check (important for Windows)
markdown = output_path.read_text(encoding="utf-8")

assert "✅" in markdown
assert "❌" in markdown

markdown.encode("utf-8")