Skip to content

Conversation

@cgwalters
Copy link
Collaborator

Previously, composefs-oci only stored the config and layer data but
discarded the OCI manifest after pulling. This forced downstream
consumers (like bootc) to implement manifest storage internally,
which doesn't really make sense.

My opinion is that while the true "composefs core" is today and
should remain logically flexible, there's absolutely not reason for
us not to have a very opinionated mechanism to store OCI. Very very
key to this is that OCI is not just container images (with tar media
types) - there's nowadays OCI artifacts which allow storing arbitrary
data easily - and I think mapping "foreign" types to OCI makes a
ton of sense.

So as part of this the composefs-oci layer also learns how to handle
non-tar layers from manifests.

  • Manifests stored at oci-manifest-{digest} with refs to config/layers
  • Named tags under streams/refs/oci/{name} act as GC roots
  • New OciImage type provides high-level access to stored images
  • pull_image() returns PullResult with both manifest and config info
  • Original pull() maintained for backward compatibility

CLI additions (cfsctl oci):

  • images - list all tagged OCI images
  • inspect <image> - show image details (arch, labels, layers, etc.)
  • tag <digest> <name> - tag an image by manifest digest
  • untag <name> - remove a tag

The storage model follows OCI semantics: everything is content-addressed,
manifests reference their components, and tags are just named pointers
to manifests. This enables offline access to all image metadata and
proper garbage collection of unreferenced content.

See-also: bootc-dev/bootc#1830

Assisted-by: OpenCode (Opus 4.5)

Update doc/repository.md and repository.rs module docs:

- Remove "experimental" disclaimer - the layout is stable
- Fix CLI flag: --path → --repo
- Fix hash length: 64 chars → 64/128 for sha256/sha512
- Add comprehensive module-level rustdoc to repository.rs with ASCII
  diagram of the repository structure
- Document images/ vs streams/ distinction and security rationale
- Expand name_stream() documentation to explain GC root behavior

Assisted-by: OpenCode (Claude claude-opus-4-5-20250514)
Signed-off-by: Colin Walters <walters@verbum.org>
Previously, composefs-oci only stored the config and layer data but
discarded the OCI manifest after pulling. This forced downstream
consumers (like bootc) to implement manifest storage internally,
which doesn't really make sense.

My opinion is that while the true "composefs core" is today and
should remain logically flexible, there's absolutely not reason for
us not to have a very opinionated mechanism to store OCI. Very very
key to this is that OCI is not just container images (with tar media
types) - there's nowadays OCI artifacts which allow storing arbitrary
data easily - and I think mapping "foreign" types to OCI makes a
ton of sense.

So as part of this the composefs-oci layer also learns how to handle
non-tar layers from manifests.

- Manifests stored at `oci-manifest-{digest}` with refs to config/layers
- Named tags under `streams/refs/oci/{name}` act as GC roots
- New `OciImage` type provides high-level access to stored images
- `pull_image()` returns `PullResult` with both manifest and config info
- Original `pull()` maintained for backward compatibility

CLI additions (cfsctl oci):
- `images` - list all tagged OCI images
- `inspect <image>` - show image details (arch, labels, layers, etc.)
- `tag <digest> <name>` - tag an image by manifest digest
- `untag <name>` - remove a tag

The storage model follows OCI semantics: everything is content-addressed,
manifests reference their components, and tags are just named pointers
to manifests. This enables offline access to all image metadata and
proper garbage collection of unreferenced content.

See-also: bootc-dev/bootc#1830

Assisted-by: OpenCode (Opus 4.5)
Signed-off-by: Colin Walters <walters@verbum.org>
@cgwalters
Copy link
Collaborator Author

Something I want to highlight in this PR that I think is quite cool and important is that this project is now more directly/obviously supporting non-container OCI types i.e. OCI artifacts.

Often when people say "OCI" they think of course about the classic tar layer container images, but that's not the only use case for OCI. And it makes an equal amount of sense to store e.g. AI model training weights or disk images in a composefs repo as it does OCI.

So again "OCI" != "OCI container images" in general.

@allisonkarlitskaya
Copy link
Collaborator

I looked briefly. This seems like pretty cool/useful stuff...

There's a bit of tension here with #218. What's the eventual plan? Import via containers-image-proxy-rs, or skip skopeo and use the cstor-rs stuff (via jsonrpc) directly?

@cgwalters
Copy link
Collaborator Author

There's a bit of tension here with #218.

The two PRs are 100% compatible AFAICS and complementary. This is just extending things so we store the manifest too etc., it's orthogonal to how we fetch the data.

What's the eventual plan? Import via containers-image-proxy-rs, or skip skopeo and use the cstor-rs stuff (via jsonrpc) directly?

I think for bootc we will likely always fetch via podman/skopeo for the near term future by default because of the "containers/image" part of container-libs (talking to registries, registries.conf auth file handling etc.) into a containers-storage, and then reflink from there into composefs (and then reflink from there into ostree if enabled).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants