-
Notifications
You must be signed in to change notification settings - Fork 30
Open
Labels
discussionTopics to talk through before actingTopics to talk through before actingenhancementNew feature or requestNew feature or request
Description
Summary
Extend k8s-tutorial to demonstrate Rucio data management across multiple storage technologies (S3, dCache) using the managed upload pattern.
Motivation
Current tutorial only demonstrates XRootD storage. Production Rucio deployments typically integrate with diverse storage backends (object stores, grid storage, distributed filesystems).
Proposed Changes
- Add S3/MinIO storage (2 instances) - cloud-native object storage
- Add dCache storage (2 instances) - grid storage with WebDAV
- Update workflow script demonstrating managed upload pattern
- Update deployment scripts to handle all storage types preserving previous behavior and opt-in feature flags for multi-storage demonstration
Workflow demonstrated:
- Upload files to source RSEs (XRD1, S3_1, DCACHE_1)
- Create datasets and containers
- Create replication rules to destination RSEs
- FTS3 orchestrates third-party copy transfers
- Monitor transfer completion
Testing
./scripts/deploy-rucio.sh --enable-all-storage
# Auto-detects and configures enabled storages
./scripts/use-rucio.shFuture Work
- Manual registration workflow (separate issue/branch for pre-existing data sources), e.g. see how PandaWMS handles this https://github.com/PanDAWMS/panda-server/blob/118b89651b6fbafeebd3bacf368cfb313142b991/pandaserver/dataservice/ddm.py#L17
- Cross-technology transfers using agreed upon protocol
- Performance benchmarking
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
discussionTopics to talk through before actingTopics to talk through before actingenhancementNew feature or requestNew feature or request