Adopt cargo-update-recipe-crates pattern, static UID/GIDs, CI improvements #196
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull request overview
This PR modernizes the Yocto Rust recipe workflow by switching all Rust recipes from the deprecated cargo-bitbake flow to OE-Core’s cargo-update-recipe-crates, moving crate metadata into generated *-crates.inc files and simplifying recipe maintenance.
Changes:
- Refactors Rust recipes to
inherit cargo cargo-update-recipe-cratesand moves crate SRC_URI/checksums into*-crates.inc. - Reworks
scripts/update-recipes.shand extendsscripts/recipe_helpers.pyto generate recipes + crates includes from upstream repos andCargo.lock. - Removes deprecated tooling (
cargo-bitbakeusage and patch-bitbake helper) and updates docs/devcontainer accordingly.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/update-recipes.sh | Rewritten to generate .bb, version .inc, and *-crates.inc directly from upstream repos + Cargo.lock. |
| scripts/recipe_helpers.py | Adds generate-crates-inc command to produce *-crates.inc from Cargo.lock. |
| scripts/patch-bitbake.py | Deleted (no longer needed with new generation approach). |
| recipes-core/iotedge/iotedge.inc | Updates patch list to use new panic strategy patch; removes old git-deps patch reference. |
| recipes-core/iotedge/iotedge-crates.inc | New generated crate URI + checksum include for iotedge. |
| recipes-core/iotedge/iotedge-1.5.35.inc | Adds IIS SRCREV pinning for this IoT Edge version. |
| recipes-core/iotedge/iot-identity-service.inc | Introduces IIS git dependency include used by IoT Edge recipes. |
| recipes-core/iotedge/files/0001-Remove-panic-abort-from-workspace-profiles.patch | New patch to set panic = 'unwind' for Yocto cross-compile compatibility. |
| recipes-core/iotedge/files/0003-Add-patch-section-for-IIS-deps.patch | Removed (replaced by IIS include approach). |
| recipes-core/aziotd/files/0001-Remove-panic.patch | Updates patch content to explicitly set panic = "unwind". |
| recipes-core/aziotd/aziotd_1.5.6.bb | Simplifies recipe and switches to cargo-update-recipe-crates + require ${BPN}-crates.inc. |
| recipes-core/aziotctl/files/0001-Remove-panic.patch | Updates patch content to explicitly set panic = "unwind". |
| recipes-core/aziotctl/aziotctl_1.5.6.bb | Simplifies recipe and switches to cargo-update-recipe-crates + require ${BPN}-crates.inc. |
| recipes-core/aziot-keys/files/0001-Remove-panic.patch | Updates patch content to explicitly set panic = "unwind". |
| recipes-core/aziot-edged/aziot-edged.inc | Updates patch list to use new panic strategy patch; removes old git-deps patch reference. |
| recipes-core/aziot-edged/aziot-edged-crates.inc | New generated crate URI + checksum include for aziot-edged. |
| recipes-core/aziot-edged/aziot-edged-1.5.35.inc | Adds IIS SRCREV pinning for this IoT Edge version. |
| recipes-core/aziot-edged/iot-identity-service.inc | Introduces IIS git dependency include used by IoT Edge recipes. |
| recipes-core/aziot-edged/files/0001-Remove-panic-abort-from-workspace-profiles.patch | New patch to set panic = 'unwind' for Yocto cross-compile compatibility. |
| recipes-core/aziot-edged/files/0003-Add-patch-section-for-IIS-deps.patch | Removed (replaced by IIS include approach). |
| docs/release.md | Updates release workflow documentation to reflect new recipe/crate generation flow. |
| .devcontainer/Dockerfile | Removes Rust toolchain + cargo-bitbake install; documents new approach. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jlian
force-pushed
the
fix/adopt-update-crates-192
branch
from
e89f0e2 to
fd49832
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 33 out of 33 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jlian
changed the title
Member
Author
|
Added the version matrix fix from #197 to this PR. The tooling ( Verified with |
Replace cargo-bitbake workflow with OE-Core's cargo-update-recipe-crates class. Crate dependencies are now declared in dedicated *-crates.inc files parsed from Cargo.lock, consistent with upstream Yocto conventions. Recipe changes: - All 5 recipes (aziotd, aziotctl, aziot-keys, aziot-edged, iotedge) inherit cargo-update-recipe-crates and require <BPN>-crates.inc - IoT Edge recipes (aziot-edged, iotedge) share iot-identity-service.inc for the ~25 IIS git sub-dependencies via SRCREV_FORMAT - IIS recipes use gitsm:// for submodule cloning - Version-specific .inc files carry IIS_SRCREV + VERSION exports Script changes: - update-recipes.sh: removed cargo-bitbake dependency; generates .bb templates and calls recipe_helpers.py generate-crates-inc to produce crate includes from Cargo.lock - recipe_helpers.py: added parse_cargo_lock() + cmd_generate_crates_inc() - Deleted patch-bitbake.py (no longer referenced) Cleanup: - Deleted 4 patch files (~5,900 lines): 0001-Remove-git-from-Cargo.patch and 0003-Add-patch-section-for-IIS-deps.patch for aziot-edged + iotedge - Dockerfile: removed cargo-bitbake install + Rust toolchain - docs/release.md: updated manual recipe workflow Validated: bitbake -p (0 errors), bitbake -e confirms correct SRCREV, IIS_SRCREV, CARGO_LOCK_SRC_DIR, and SRCREV_FORMAT resolution.
- Write placeholder file when no registry crates found in Cargo.lock instead of returning early (avoids stale crates.inc files) - Use _is_registry_source() helper to properly handle sparse+ and other non-git/path Cargo registry protocols - Add curl to dependency check in update-recipes.sh - Deduplicate iot-identity-service.inc into a single shared file at recipes-core/iot-identity-service.inc with IIS_GIT_URI variable, referenced via BBPATH from both aziot-edged and iotedge recipes
- Add check-recipes job to ci-build.yml (ubuntu-latest) that regenerates recipes from pinned versions and diffs against committed files. Build job depends on this passing. - Add bitbake -p parse-check to bitbake.sh before the actual build so every build (CI and local) catches recipe syntax errors early.
Assign static UID/GID values to iotedge (13620), aziotcs (13624), aziotks (13625), aziotid (13626), and aziottpm (13627) so they remain consistent across builds. This is important for A/B partition schemes and [[principal]] config references in service TOML files. The docker group is left dynamic since other layers may also create it. edgeagentuser (13622) and edgehubuser (13623) were already static. Verified in QEMU: all UIDs/GIDs match expected values in /etc/passwd and /etc/group, services start correctly. Fixes: #130
- Remove placeholder index hashes (a1b2c3d) from IIS panic patches - Move dependency check before first curl/git/python3 usage in update-recipes.sh - Guard check-recipes job with docs_only condition to skip on docs-only PRs - Reorder status-check to evaluate docs_only before check-recipes result
The release tag (e.g. 1.5.35) may only update Docker images while the
daemon binaries (aziot-edged, iotedge) stay at an earlier version
(e.g. 1.5.21 per product-versions.json). Using the release tag as the
recipe version caused runtime failures:
Unable to find image 'mcr.microsoft.com/azureiotedge-diagnostics:1.5.35'
Changes:
update-recipes.sh:
- Extracts 'aziot-edge' component version (daemon version) from
product-versions.json and uses it for recipe filenames, VERSION
export, and SRCREV resolution
- Stores the release tag as IOTEDGE_RELEASE in the version .inc
file for traceability
check-upstream.sh:
- Reads IOTEDGE_RELEASE from the .inc file to determine if the
current release is already tracked, preventing false 'docker-only'
notifications when the release is already handled
- Outputs current_release and update_type for workflow use
ci-build.yml:
- check-recipes job reads IOTEDGE_RELEASE from .inc to determine
which release tag to pass to update-recipes.sh for idempotency
watch-upstream.yml:
- notify-docker-only condition uses update_type instead of comparing
recipe filename version against release version
- Adds current_release and update_type outputs
Fixes #197
jlian
force-pushed
the
fix/adopt-update-crates-192
branch
from
c5eaa19 to
9754c72
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 35 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
scripts/update-recipes.sh:128
- Same tag-SHA resolution issue for IIS:
git ls-remote --tags ... "refs/tags/${IIS_VERSION}"won’t peel annotated tags. Prefer resolvingrefs/tags/${IIS_VERSION}^{}(or call the existingrecipe_helpers.py tag-shahelper) soprepare_repoalways checks out a commit SHA.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- watch-upstream.yml: Use current_release (not current_recipe) for release
row in automated PR body table
- bitbake.sh: Check bitbake -p exit code and fail fast on parse errors
instead of printing success unconditionally
- update-recipes.sh: Resolve peeled commit SHA for annotated tags by
querying refs/tags/<ver>^{} first, falling back to lightweight tag ref
ggjjj
approved these changes
Feb 11, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adopts the modern
cargo-update-recipe-cratesBitBake class for all five Rust recipes, replacing the deprecatedcargo-bitbakeworkflow. Closes #192.Also adds static UIDs/GIDs for all IoT Edge system users and groups. Fixes #130.
Credit to @PeteDavidson for pioneering this approach in #184.
Changes
Static UIDs/GIDs (#130)
All aziot system users and groups now receive deterministic IDs in the 136xx range, ensuring consistency across builds (important for A/B partition schemes and
[[principal]]config references):iotedgeedgeagentuseredgehubuseraziotcsaziotksaziotidaziottpmThe
dockergroup is left dynamic because other layers may also create it. IDs can be overridden via.bbappendwith customUSERADD_PARAM/GROUPADD_PARAM.Recipe refactoring
iotedge,aziot-edged,aziotd,aziotctl,aziot-keys) nowinherit cargo cargo-update-recipe-crates.bbfiles into separate*-crates.incfiles (generated fromCargo.lock)*-<version>.incfilesiot-identity-service.inc(used byiotedgeandaziot-edged)Patches
0001-Remove-git-from-Cargo.patch(2,929 lines each for iotedge and aziot-edged) —cargo_common_do_patch_pathsnow handles Cargo.lock git source stripping natively0003-Add-patch-section-for-IIS-deps.patch— replaced byiot-identity-service.incpanic = 'unwind'instead of removing the line, for cross-compilation compatibility with Yocto's Rust sysroot (approach from @PeteDavidson's Changes for version 1.5.24 (aziotctl at 1.5.6) #184)Tooling
scripts/update-recipes.shto generate.bb, version.inc, and-crates.incfiles from upstream reposgenerate_crates_inc()toscripts/recipe_helpers.pyfor crate file generation fromCargo.lockscripts/patch-bitbake.py(no longer needed)cargo-bitbake/ Rust toolchain from.devcontainer/Dockerfiledocs/release.mdwith new workflowCI improvements
check-recipesjob toci-build.ymlthat regenerates recipes from pinned versions and verifies committed files match (idempotency check). Runs onubuntu-latest(~2 min) and gates the build job.bitbake -pparse-check toscripts/bitbake.shso every build (CI and local) validates recipe syntax before starting the expensive compile.Future updates
Recipe crates can now be refreshed via:
Or regenerated from scratch via:
Validation
bitbake -p— 0 parse errorsbitbake iotedge aziot-edged— full build succeededupdate-recipes.shround-trip — generated files match committed files exactly/etc/passwdand/etc/groupVersion matrix fix (#197)
The recipe generation logic now correctly distinguishes between the upstream release tag (e.g.,
1.5.35) and the daemon binary version (e.g.,1.5.21) fromproduct-versions.json. Some releases only update Docker images while the daemon binaries stay at an earlier version. Previously, recipes used the release tag for everything, causing runtime failures when the daemon tried to pull container images that don't exist at that tag.update-recipes.shextracts theaziot-edgecomponent version fromproduct-versions.jsonand uses it for recipe filenames,VERSIONexport, andSRCREVIOTEDGE_RELEASEin the version.incfile for traceabilitycheck-upstream.shcomparesIOTEDGE_RELEASE(not recipe filename) against upstream to avoid false docker-only notificationsci-build.ymlreadsIOTEDGE_RELEASEfrom the.incfile for the idempotency checkwatch-upstream.ymluses the newupdate_typeoutput for the docker-only notification conditionrelease.mdupdated with version detection docs and tagging notesThis is the root-cause fix for the issue identified in #197. If #197 is merged first as a quick manual fix, this branch will be rebased on top.