Skip to content

Conversation

@Jagriti-student
Copy link
Contributor

@Jagriti-student Jagriti-student commented Dec 19, 2025

Added version attribute to agentunit package.
Version matches pyproject.toml.

closes #11

Summary by CodeRabbit

  • Documentation

    • Added guidance for running CI checks locally using ruff and pytest.
  • New Features

    • Expanded public API to expose Runner, Scenario, ScenarioResult, SuiteResult and run_suite.
  • Tests

    • Added tests validating Markdown report generation and UTF-8 encoding.
  • Chores

    • Included a sample generated report demonstrating pass/fail output.

✏️ Tip: You can customize this high-level summary in your review settings.

@continue
Copy link

continue bot commented Dec 19, 2025

All Green - Keep your PRs mergeable

Learn more

All Green is an AI agent that automatically:

✅ Addresses code review comments

✅ Fixes failing CI checks

✅ Resolves merge conflicts


Unsubscribe from All Green comments

@coderabbitai
Copy link

coderabbitai bot commented Dec 19, 2025

Walkthrough

Adds local CI instructions to README, extends public exports in src/agentunit/__init__.py, adds a reporting unit test that asserts emoji presence and UTF‑8 encoding, and adds a sample report.md.

Changes

Cohort / File(s) Summary
Documentation
README.md
Added "Running CI Checks Locally" subsection with commands to run ruff and pytest locally (mirrors CI steps).
Public API / Package init
src/agentunit/__init__.py
Extended __all__ to re-export Runner, Scenario, ScenarioResult, SuiteResult, and run_suite; moved module docstring location (cosmetic).
Tests
tests/test_reporting.py
New unit test validating SuiteResult.to_markdown includes ✅ and ❌ and is UTF‑8 encodable.
Artifacts
report.md
Added generated Markdown report content showing a 50% success rate and two test outcomes (✅ / ❌).

Sequence Diagram(s)

(omitted — changes are documentation, API re-exports, and a unit test; no new multi-component control flow)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers

  • aviralgarg05

Pre-merge checks and finishing touches

❌ Failed checks (3 warnings, 1 inconclusive)
Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR summary indicates changes to init.py extending the public API with new exports (Runner, Scenario, ScenarioResult, SuiteResult, run_suite), adding tests, and documentation, but issue #11 only requires adding version attribute to src/agentunit/init.py. Verify whether the extensive all modifications and new test file are required by issue #11 or represent scope creep. If not required, remove unrelated changes to focus on the version objective.
Out of Scope Changes check ⚠️ Warning The changeset includes multiple modifications beyond the version requirement: expanded all exports, test file additions (test_reporting.py), markdown report generation, and README updates that are not mentioned in issue #11's objectives. Remove out-of-scope changes including all expansion, test_reporting.py, and report.md generation. Retain only the version addition required by issue #11.
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
Description check ❓ Inconclusive The PR description is minimal but covers the essential change: adding version attribute to match pyproject.toml and references the linked issue #11. However, it omits most template sections required for comprehensive documentation. Consider adding sections from the template such as Type of Change, Changes Made, Testing, and Code Quality to provide more complete documentation of the implementation.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Expose version in agentunit package' accurately describes the main objective referenced in issue #11, which is to expose the version attribute in the package.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
src/agentunit/__init__.py (1)

21-22: Implementation approved; version matches pyproject.toml.

The __version__ attribute is correctly exposed and meets the acceptance criteria (import agentunit; print(agentunit.__version__)).

Consider using importlib.metadata to access metadata of an installed Distribution Package, which would eliminate the need to manually keep this in sync with pyproject.toml:

from importlib.metadata import version

__version__ = version("agentunit")
tests/test_reporting.py (2)

5-6: Avoid manual sys.path manipulation in tests.

Modifying sys.path is fragile and non-standard for pytest. Instead, ensure the package is installed in development mode (pip install -e .) or configure pytest's pythonpath in pyproject.toml.

🔎 Suggested approach

Remove lines 5-6 and configure pytest properly:

# In pyproject.toml
[tool.pytest.ini_options]
pythonpath = ["src"]

Or install the package in editable mode:

pip install -e .

37-38: UTF-8 encoding check should verify no exception is raised.

The encode("utf-8") call discards its result. While calling it will raise if encoding fails, the intent would be clearer with an explicit assertion or a comment explaining the check.

🔎 Suggested improvement
     # UTF-8 safety
-    markdown.encode("utf-8")
+    # Verify markdown can be encoded as UTF-8 (raises UnicodeEncodeError if it cannot)
+    _ = markdown.encode("utf-8")

Or use a try/except to make the assertion explicit:

    # UTF-8 safety
    try:
        markdown.encode("utf-8")
    except UnicodeEncodeError:
        pytest.fail("Markdown output is not valid UTF-8")
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b85e986 and 75729b2.

📒 Files selected for processing (11)
  • README.md (1 hunks)
  • src/agentunit/__init__.py (1 hunks)
  • src/agentunit/adapters/base.py (1 hunks)
  • src/agentunit/core/__init__.py (1 hunks)
  • src/agentunit/core/exceptions.py (1 hunks)
  • src/agentunit/core/replay.py (2 hunks)
  • src/agentunit/core/runner.py (1 hunks)
  • src/agentunit/core/scenario.py (4 hunks)
  • src/agentunit/core/trace.py (3 hunks)
  • src/agentunit/py.typed (0 hunks)
  • tests/test_reporting.py (1 hunks)
💤 Files with no reviewable changes (1)
  • src/agentunit/py.typed
🧰 Additional context used
🧬 Code graph analysis (2)
tests/test_reporting.py (4)
src/agentunit/reporting/results.py (1)
  • SuiteResult (70-129)
src/agentunit/comparison/comparator.py (1)
  • RunResult (21-43)
src/agentunit/datasets/base.py (1)
  • name (38-39)
src/agentunit/collaboration/branch.py (1)
  • status (173-195)
src/agentunit/adapters/base.py (3)
tests/test_runner.py (2)
  • execute (30-38)
  • cleanup (40-41)
src/agentunit/datasets/base.py (1)
  • DatasetCase (19-27)
src/agentunit/core/trace.py (1)
  • TraceLog (26-78)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Test (Python 3.10)
🔇 Additional comments (7)
src/agentunit/core/runner.py (1)

1-3: LGTM!

The multi-line docstring format is consistent with Python documentation conventions.

src/agentunit/core/scenario.py (1)

1-3: LGTM!

Docstring reformatting to multi-line format improves readability and follows Python documentation conventions consistently across the module.

Also applies to: 24-26, 82-84, 100-102

src/agentunit/core/exceptions.py (1)

1-23: LGTM!

Docstring reformatting is consistent with the rest of the PR and follows Python documentation conventions.

src/agentunit/core/trace.py (1)

1-3: LGTM!

The multi-line docstring reformatting improves readability and is consistent across the module.

Also applies to: 16-18, 27-29

src/agentunit/core/__init__.py (1)

1-3: LGTM!

The multi-line docstring formatting is consistent with other modules in this PR.

src/agentunit/core/replay.py (1)

1-3: LGTM!

The multi-line docstring formatting is consistent with the pattern applied across the codebase in this PR.

Also applies to: 13-15

src/agentunit/adapters/base.py (1)

35-42: LGTM!

The expanded docstrings with Args and Returns sections improve API documentation for implementers of the BaseAdapter interface.

Also applies to: 46-55, 58-66

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/agentunit/__init__.py 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/agentunit/__init__.py (1)

21-21: Consider using importlib.metadata to avoid version drift.

The hardcoded version string requires manual updates whenever the version changes in pyproject.toml, creating a maintenance burden and risk of version drift. Issue #11 mentioned importlib.metadata as an alternative that automatically reads from package metadata.

🔎 Proposed refactor using importlib.metadata
+from importlib.metadata import version, PackageNotFoundError
+
+try:
+    __version__ = version("agentunit")
+except PackageNotFoundError:
+    # Fallback for development installs
+    __version__ = "0.0.0+dev"
-__version__ = "0.7.0"

This approach eliminates the need to manually sync versions between pyproject.toml and the source code.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between af8264b and 205c8ba.

📒 Files selected for processing (3)
  • report.md
  • src/agentunit/__init__.py
  • tests/test_reporting.py
✅ Files skipped from review due to trivial changes (1)
  • report.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/test_reporting.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Test (Python 3.10)

Comment on lines +9 to +10
"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Move the module docstring to the correct position.

Module docstrings must be the first statement in a file (immediately after from __future__ imports). When placed after regular imports, the string becomes a discarded literal and won't be accessible via help(agentunit) or agentunit.__doc__.

🔎 Proposed fix
 from __future__ import annotations
 
+"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""
+
 from .core.runner import Runner, run_suite
 from .core.scenario import Scenario
 from .datasets.base import DatasetCase, DatasetSource
 from .reporting.results import ScenarioResult, SuiteResult
 
 
-"""AgentUnit - pytest-style evaluation harness for agentic AI and RAG workflows."""
-
 __all__ = [
     "DatasetCase",
     "DatasetSource",
🤖 Prompt for AI Agents
In src/agentunit/__init__.py around lines 9-10, the module docstring is placed
after regular imports so it becomes a discarded literal; move the triple-quoted
docstring to be the very first statement in the file (immediately after any from
__future__ imports if present) so it becomes the module-level docstring
accessible via help() and __doc__.

@Jagriti-student
Copy link
Contributor Author

I have solve the issue.

Copy link
Owner

@aviralgarg05 aviralgarg05 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@aviralgarg05 aviralgarg05 merged commit 030bf99 into aviralgarg05:main Jan 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose __version__ in agentunit package

3 participants