-
Notifications
You must be signed in to change notification settings - Fork 213
feat: report bundle, presets, tiling+blend, metrics; tests, docs, CI #95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,35 +1,94 @@ | ||
| ## Description | ||
| Brief description of what this PR does. | ||
| ## 🚀 Feature Summary | ||
|
|
||
| ## Changes Made | ||
| - [ ] List the main changes here | ||
| This PR implements **4 key features** for DreamLayer AI: | ||
|
|
||
| ## Evidence Required ✅ | ||
| 1. **Optional Metrics Support** - CLIP and LPIPS gracefully fallback when dependencies missing | ||
| 2. **Test Suite Optimization** - Heavy metric tests auto-skip without torch/transformers/lpips | ||
| 3. **Deterministic Bundles** - Fixed ZIP timestamps ensure identical SHA256 for same content | ||
| 4. **CI/CD Integration** - GitHub Actions runs tests without heavy dependencies | ||
|
|
||
| ### UI Screenshot | ||
| <!-- Paste a screenshot of the UI changes here --> | ||
| ![UI Screenshot]() | ||
| ## 🧪 Test Strategy | ||
|
|
||
| ### Generated Image | ||
| <!-- Paste an image generated with your changes here --> | ||
| ![Generated Image]() | ||
| ### Test Coverage | ||
| - ✅ **SSIM tests**: Always enabled (lightweight scikit-image dependency) | ||
| - ✅ **CLIP tests**: Auto-skip with `@pytest.mark.requires_torch` when torch/transformers missing | ||
| - ✅ **LPIPS tests**: Auto-skip with `@pytest.mark.requires_lpips` when lpips missing | ||
| - ✅ **Fallback behavior**: All tests verify graceful degradation returns `None` values | ||
|
|
||
| ### Logs | ||
| <!-- Paste relevant logs that verify your changes work --> | ||
| ```text | ||
| # Paste logs here | ||
| ### Test Execution | ||
| ```bash | ||
| # Run all tests (heavy deps auto-skip) | ||
| python -m pytest tests/ --tb=short -q | ||
|
|
||
| # Verify specific metric behavior | ||
| python -m pytest tests/test_quality_metrics.py -v | ||
| ``` | ||
|
|
||
| ## 🔒 Determinism Notes | ||
|
|
||
| - **ZIP timestamps**: Fixed to `(1980,1,1,0,0,0)` for reproducible SHA256 | ||
| - **Bundle verification**: Two identical runs produce identical hashes | ||
| - **Content integrity**: SHA256 verification ensures bundle consistency | ||
|
|
||
| ## 📦 Optional Dependencies Behavior | ||
|
|
||
| | Dependency | Status | Test Behavior | | ||
| |------------|--------|---------------| | ||
| | `torch + transformers` | Optional | CLIP tests auto-skip | | ||
| | `lpips` | Optional | LPIPS tests auto-skip | | ||
| | `scikit-image` | Required | SSIM tests always run | | ||
|
|
||
| **Graceful fallbacks**: When optional deps missing, metrics return `None` instead of crashing. | ||
|
|
||
| ## 🎯 Instructions to Reproduce | ||
|
|
||
| ### 1. Test Heavy Dependency Skipping | ||
| ```bash | ||
| # Install without heavy deps | ||
| pip install -r requirements.txt | ||
| # (torch/transformers/lpips not installed) | ||
|
|
||
| # Run tests - heavy tests should auto-skip | ||
| python -m pytest tests/ -q | ||
| ``` | ||
|
|
||
| ### 2. Verify Deterministic Bundle | ||
| ```bash | ||
| # Generate two bundles from same run | ||
| python dream_layer.py --report-bundle --report-out ./bundle1.zip | ||
| python dream_layer.py --report-bundle --report-out ./bundle2.zip | ||
|
|
||
| # Verify identical SHA256 | ||
| sha256sum bundle1.zip bundle2.zip | ||
| # Should produce identical hashes | ||
| ``` | ||
|
|
||
| ### Tests (Optional) | ||
| <!-- If you added tests, paste the test results here --> | ||
| ```text | ||
| # Test results | ||
| ### 3. Test Metric Fallbacks | ||
| ```bash | ||
| # Without CLIP/LPIPS deps, metrics return None | ||
| python -c " | ||
| from metrics.clip_score import clip_text_image_similarity | ||
| from PIL import Image | ||
| img = Image.new('RGB', (100, 100)) | ||
| scores = clip_text_image_similarity([img], ['test']) | ||
| print(f'CLIP scores: {scores}') # Should be [None] | ||
| " | ||
| ``` | ||
|
|
||
| ## Checklist | ||
| - [ ] UI screenshot provided | ||
| - [ ] Generated image provided | ||
| - [ ] Logs provided | ||
| - [ ] Tests added (optional) | ||
| - [ ] Code follows project style | ||
| - [ ] Self-review completed | ||
| ## 🔍 Code Quality | ||
|
|
||
| - **Pre-commit hooks**: black, isort, flake8 for consistent formatting | ||
| - **Type hints**: Full type annotations for maintainability | ||
| - **Error handling**: Graceful fallbacks throughout metrics pipeline | ||
| - **Documentation**: Comprehensive README updates with examples | ||
|
|
||
| ## 📋 Checklist | ||
|
|
||
| - [x] Tests pass without heavy dependencies | ||
| - [x] CLIP/LPIPS tests auto-skip when deps missing | ||
| - [x] SSIM tests remain always enabled | ||
| - [x] Bundle determinism verified | ||
| - [x] Pre-commit hooks configured | ||
| - [x] CI workflow added | ||
| - [x] Documentation updated | ||
| - [x] Code follows project style guidelines |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,38 @@ | ||||||||||||||||||||||||||||||||||||||||||||||||||
| name: Test Suite | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
| on: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| push: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| branches: [ main, develop ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
| pull_request: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| branches: [ main, develop ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
| jobs: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| test: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| runs-on: ubuntu-latest | ||||||||||||||||||||||||||||||||||||||||||||||||||
| strategy: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| matrix: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| python-version: [3.8, 3.9, "3.10", "3.11"] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
| steps: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| - uses: actions/checkout@v4 | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: Set up Python ${{ matrix.python-version }} | ||||||||||||||||||||||||||||||||||||||||||||||||||
| uses: actions/setup-python@v4 | ||||||||||||||||||||||||||||||||||||||||||||||||||
| with: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| python-version: ${{ matrix.python-version }} | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: Install dependencies | ||||||||||||||||||||||||||||||||||||||||||||||||||
| run: | | ||||||||||||||||||||||||||||||||||||||||||||||||||
| python -m pip install --upgrade pip | ||||||||||||||||||||||||||||||||||||||||||||||||||
| python -m pip install -r requirements.txt | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+17
to
+28
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion Update GitHub Actions to latest versions for security and compatibility The workflow is using outdated action versions that may have security vulnerabilities or compatibility issues with newer GitHub runners. Apply this diff to update to the latest stable versions: - - uses: actions/checkout@v4
+ - uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
- uses: actions/setup-python@v4
+ uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -r requirements.txt📝 Committable suggestion
Suggested change
🧰 Tools🪛 actionlint (1.7.7)20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue (action) 🪛 YAMLlint (1.37.1)[error] 18-18: trailing spaces (trailing-spaces) [error] 23-23: trailing spaces (trailing-spaces) [error] 28-28: trailing spaces (trailing-spaces) 🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: Run tests (no heavy deps) | ||||||||||||||||||||||||||||||||||||||||||||||||||
| run: | | ||||||||||||||||||||||||||||||||||||||||||||||||||
| cd dream_layer_backend | ||||||||||||||||||||||||||||||||||||||||||||||||||
| python -m pytest tests/ -q --tb=short | ||||||||||||||||||||||||||||||||||||||||||||||||||
| # Heavy metric tests should auto-skip when torch/transformers/lpips not installed | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: Verify test coverage | ||||||||||||||||||||||||||||||||||||||||||||||||||
| run: | | ||||||||||||||||||||||||||||||||||||||||||||||||||
| cd dream_layer_backend | ||||||||||||||||||||||||||||||||||||||||||||||||||
| python -m pytest tests/ --collect-only -q | grep -E "(CLIP|LPIPS)" || echo "No heavy metric tests found (expected)" | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| exclude: | | ||
| (?x)( | ||
| ^runs/| | ||
| ^dist/| | ||
| ^build/| | ||
| ^\.venv/| | ||
| ^env/| | ||
| ^\.mypy_cache/| | ||
| ^\.pytest_cache/| | ||
| ^docs/_build/ | ||
| ) | ||
|
|
||
| repos: | ||
| - repo: https://github.com/pre-commit/pre-commit-hooks | ||
| rev: v4.6.0 | ||
| hooks: | ||
| - id: trailing-whitespace | ||
| - id: end-of-file-fixer | ||
| - id: check-yaml | ||
| - id: check-json | ||
| - id: check-toml | ||
| - id: check-ast | ||
| - id: check-added-large-files | ||
| - id: check-merge-conflict | ||
| - id: debug-statements | ||
| - id: name-tests-test | ||
| - id: mixed-line-ending | ||
| args: ["--fix=lf"] | ||
|
|
||
| - repo: https://github.com/psf/black-pre-commit-mirror | ||
| rev: 24.4.2 | ||
| hooks: | ||
| - id: black | ||
| language_version: python3 | ||
| args: [--line-length=120] | ||
|
|
||
| - repo: https://github.com/PyCQA/isort | ||
| rev: 5.13.2 | ||
| hooks: | ||
| - id: isort | ||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| args: ["--profile", "black", "--line-length=120"] | ||
|
|
||
| - repo: https://github.com/PyCQA/flake8 | ||
| rev: 7.0.0 | ||
| hooks: | ||
| - id: flake8 | ||
| args: [--config=.flake8] | ||
|
|
||
| - repo: https://github.com/astral-sh/ruff-pre-commit | ||
| rev: v0.4.8 | ||
| hooks: | ||
| - id: ruff | ||
| args: ["--line-length=120"] | ||
|
|
||
| - repo: https://github.com/asottile/pyupgrade | ||
| rev: v3.19.1 | ||
| hooks: | ||
| - id: pyupgrade | ||
| args: [--py39-plus] | ||
|
|
||
| - repo: https://github.com/codespell-project/codespell | ||
| rev: v2.3.0 | ||
| hooks: | ||
| - id: codespell | ||
| args: | ||
| - --skip=.git,*.lock,*.csv,*.tsv,*.ipynb,*.svg,*.png,*.jpg,*.jpeg,*.gif,*.pdf | ||
| - --ignore-words-list=nd,crate,fo,te | ||
| - --quiet-level=2 | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update setup-python to v5 to avoid deprecation issues
actionlint flags v4 as too old for current runners. Upgrade to v5 to stay supported.
Apply this diff:
📝 Committable suggestion
🧰 Tools
🪛 actionlint (1.7.7)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🤖 Prompt for AI Agents