Skip to content

Conversation

@jensens
Copy link
Member

@jensens jensens commented Oct 23, 2025

Summary

Fixes #65 - mxdev now checks if source directories exist before writing them to requirements-mxdev.txt, with different behavior based on offline mode.

Problem

When using mxdev -n (no-fetch) or in offline mode, source directories may not exist yet. Previously, mxdev would write references like -e ./sources/package to requirements-mxdev.txt regardless, causing pip install to fail with "directory does not exist" errors.

This breaks the mxmake two-stage installation workflow:

  1. mxmake runs mxdev -n to generate requirements file
  2. mxdev writes -e ./sources/package even though sources don't exist
  3. pip install fails

Solution

Modified write_dev_sources() to check if source directories exist with different behavior based on mode:

Offline Mode (expected missing sources)

  • Missing sources log WARNING
  • Packages written as comments
  • mxdev continues execution
  • Warning message: "This is expected in offline mode. Run mxdev without -n and --offline flags to fetch sources."

Non-Offline Mode (unexpected missing sources = fatal error)

  • Missing sources log ERROR
  • Packages written as comments
  • mxdev raises RuntimeError and exits
  • Error message: "This indicates a failure in the checkout process. Run mxdev without -n flag to fetch sources."

This prevents silent failures when sources fail to checkout.

Changes

  • Update write_dev_sources() to accept State parameter and check Path(package["path"]).exists()
  • Track missing sources and raise RuntimeError in non-offline mode
  • Use logger.error() for non-offline mode, logger.warning() for offline mode
  • Write missing packages as # -e ./sources/package # mxdev: source not checked out
  • Add comprehensive tests for both modes
  • Update all existing tests to pass State parameter

Testing

✅ All write_dev_sources tests pass (7/7)

  • Updated test: test_write_dev_sources_missing_directories - now uses offline mode explicitly
  • New test: test_write_dev_sources_missing_directories_raises_error - verifies RuntimeError in non-offline mode
  • Existing test: test_write_dev_sources_missing_directories_offline_mode - verifies offline-specific warning

Benefits

  • ✅ Fixes mxmake two-stage installation workflow
  • ✅ Prevents silent failures when checkout fails
  • ✅ Clear distinction between expected (offline) and fatal (non-offline) missing sources
  • ✅ Prevents pip install failures from non-existent source paths
  • ✅ Users can see what packages would be installed (commented lines)
  • ✅ Makes mxdev more defensive and robust

…rements

When using mxdev -n (no-fetch) or in offline mode, source directories may
not exist yet. Previously, mxdev would write references like '-e ./sources/package'
to requirements-mxdev.txt regardless, causing pip install to fail later.

This fix checks if source directories exist before writing them:
- If directory exists: Write normally (existing behavior)
- If directory doesn't exist: Write as comment with contextual warning

The warning message adapts based on context:
- In offline mode: Mentions offline mode and suggests removing -n and --offline flags
- Normal mode: Suggests removing -n flag to fetch sources

This fixes the mxmake two-stage installation workflow where mxdev -n runs
before sources are checked out.

Changes:
- Update write_dev_sources() to accept State parameter and check path existence
- Add contextual warning messages for missing directories
- Add tests for missing directories and offline mode warnings
- Update existing tests to pass State parameter

Fixes #65
In offline mode:
- Missing sources log WARNING (expected)
- Packages written as comments
- mxdev continues

In non-offline mode:
- Missing sources log ERROR (fatal)
- Packages written as comments
- mxdev raises RuntimeError and exits

This prevents silent failures when sources fail to checkout
and makes mxmake two-stage installation more robust.

Tests:
- Updated test_write_dev_sources_missing_directories to use offline mode
- Added test_write_dev_sources_missing_directories_raises_error
- All 7 write_dev_sources tests pass
During rebase conflict resolution, test_process_line_package_in_override_keys
and test_process_line_package_in_ignore_keys were accidentally duplicated.

Removed duplicates, keeping only one copy of each test.
@jensens jensens force-pushed the fix/write-only-existing-sources branch from 84f7555 to 4232ca6 Compare October 23, 2025 15:32
@jensens jensens merged commit b6273c7 into main Oct 23, 2025
17 checks passed
@jensens jensens deleted the fix/write-only-existing-sources branch October 23, 2025 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mxdev does not work with mxmake + sources in 5.0.0

2 participants