chore(repo): Add smoke tests, create inventory, and organize notebooks

[Copilot]

Overview

This PR implements repository maintenance tasks to improve code quality and CI reliability:

• Adds smoke tests to validate app imports without full UI rendering
• Creates a comprehensive repository inventory documenting cleanup opportunities
• Reorganizes experimental notebooks into dedicated directory
• Adds helper script for ongoing code quality analysis

Changes

1. Smoke Test Suite (tests/test_smoke_app.py)

Added 5 smoke tests that validate core application imports work correctly without network calls or full Streamlit rendering:

# Validates critical import paths
test_app_imports_successfully()              # app.py module
test_app_logic_imports_successfully()        # src.app_logic
test_app_logic_initialize_without_streamlit()  # initialization logic
test_core_imports_successfully()             # src.core
test_ai_service_imports_successfully()       # src.ai_service

These tests use proper mocking (including a custom SessionStateMock class) to handle Streamlit's session state without requiring the full Streamlit runtime. All tests pass:

$ pytest -k smoke -v
# 5 passed, 55 deselected in ~6 seconds ✅

2. GitHub Actions CI Integration

Updated .github/workflows/ci.yml to run smoke tests before unit tests, ensuring app import validation in CI:

- name: Run smoke tests
  run: |
    pytest -k smoke -v
  env:
    PYTHONPATH: ${{ github.workspace }}

3. Repository Inventory (REPO_INVENTORY.md)

Created comprehensive inventory documenting:

• Actions taken: Notebook reorganization with justification
• Files under review: Configuration consolidation opportunities (.replit, pyproject.toml/setup.cfg)
• Documentation assessment: All docs reviewed, noted relationship between ARCHITECTURE.md and ARCHITECTURE_V2.md
• Code quality notes: Cleanup recommendations with risk assessment
• Future strategy: Guidance for ongoing maintenance

4. Notebook Organization

Moved experimental notebook to dedicated archive:

• Created notebooks/experiments/ directory
• Moved notebooks/pipeline_csv_to_parquet multifile.ipynb → notebooks/experiments/
• Non-destructive preservation for historical reference (36KB experimental notebook)

5. Cleanup Helper Script (scripts/report_unused.sh)

Added executable bash script for ongoing code quality analysis:

• Integrates with vulture for unused code detection (optional, with --install flag)
• Provides coverage analysis highlighting untested code
• Checks for cleanup patterns (_old files, TODO markers)
• Includes helpful tips and references REPO_INVENTORY.md

$ bash scripts/report_unused.sh
# Provides comprehensive code quality report

Testing

All existing tests continue to pass:

• ✅ Smoke tests: 5 passed
• ✅ Unit tests: 49 passed
• ✅ Code formatting: Black and isort compliant
• ✅ Linting: flake8 clean

Impact

• No behavioral changes to the application
• Additive only - all changes are non-destructive
• Improved CI reliability - smoke tests catch import issues early
• Better maintainability - inventory and helper script aid future cleanup

Rollback

All changes are easily reversible:

• Moved notebook preserved in git history
• New files can be removed without impact
• CI workflow change is additive only
• Simple git revert restores previous state

---

Resolves the repository maintenance tasks outlined in the issue. Ready for review and merge! 🚀

Original prompt

---

This section details on the original issue you should resolve

<issue_title>chore(repo): inventory unused files + add app smoke test</issue_title>
<issue_description>Goal

• Create a repo inventory of files that appear unused, large notebooks, and high-risk items.
• Add a minimal smoke test that starts Streamlit app import (no UI rendering) to assert app imports successfully.

Tasks

• Run static analysis and produce REPO_INVENTORY.md at repo root listing: files to remove, move, or archive.
• Move notebooks/pipeline_csv_to_parquet multifile.ipynb -> notebooks/experiments/ (do not delete).
• Add tests/test_smoke_app.py that imports src.app_logic or app.py initialization path (no network calls).
• Add scripts/report_unused.sh (vulture + coverage) to help subsequent cleanup.

Acceptance criteria

• REPO_INVENTORY.md created and reviewed.
• pytest -k smoke passes locally.
• No behavioral changes to app.
  Rollback
• Revert branch; inventory is non-destructive.</issue_description>

<agent_instructions>Add smoke test to github action and it shoudl pass... </agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes #6

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 22.87%. Comparing base (b011662) to head (9a5431e).

Additional details and impacted files

@@           Coverage Diff           @@
##             main       #7   +/-   ##
=======================================
  Coverage   22.87%   22.87%           
=======================================
  Files          24       24           
  Lines        1893     1893           
  Branches      330      330           
=======================================
  Hits          433      433           
  Misses       1415     1415           
  Partials       45       45           

Flag	Coverage Δ
unittests	22.87% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.