Cloud Storage Slim

Make operations across various cloud storage platforms simple.

Installation

pip install cloud-storage-slim

Usage

from cloud_storage_slim import CloudStorageSlim

cloud_storage = CloudStorageSlim()
cloud_storage.copyto('gs://bucket1/object1', 'az://bucket2/object2')

Features

• copyto
• ls

Supported Cloud Storage

• Google Cloud Storage
• Azure Blob Storage
• AlibabaCloud / Aliyun OSS
• AWS S3
• Byteplus & Volcengine - Torch Object Storage

Credentials

Google Cloud Storage

• How Application Default Credentials works
• GOOGLE_APPLICATION_CREDENTIALS
• google-cloud-python

pip install google-cloud-storage

Amazon S3

• Boto3 Configuring credentials
• Using environment variables
• AWS_ENDPOINT_URL_S3
• AWS_DEFAULT_REGION
• AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN
• Boto3

pip install boto3

Azure Blob Storage

• Manage storage account access keys
• AZURE_STORAGE_ACCOUNT_NAME, AZURE_STORAGE_ACCOUNT_KEY

pip install azure-storage-blob azure-identity

AlibabaCloud / Aliyun OSS

• Configure access credentials
• OSS_ENDPOINT
• OSS_ACCESS_KEY_ID, OSS_ACCESS_KEY_SECRET
• OSS_SESSION_TOKEN
• oss2

pip install oss2

Byteplus & Volcengine - Torch Object Storage

• Byteplus Region and Endpoint
• Volcengine Region and Endpoint

S3 Compatible API

• Byteplus Compatibility with Amazon S3
• Volcengine Compatibility with Amazon S3

TOS SDK

• TOS_ENDPOINT_URL
• TOS_ACCESS_KEY_ID TOS_SECRET_ACCESS_KEY
• TOS_DEFAULT_REGION
• ve-tos-python-sdk

pip install tos

Test cases

before run test cases, you need to prepare the test bucket.