support object storage (s3,gcs) and more #6
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces major enhancements to the caching library, focusing on new storage backends, improved documentation, and robust CI/integration test infrastructure. The most significant updates are the addition of object storage backends (S3, GCS, LocalFileCache), a multi-level
ChainCache, deduplication support for writes, and thorough documentation and testing for production use cases.Major features and improvements:
Storage Backends and Caching Architecture
ChainCachefor multi-level, composable cache hierarchies (e.g., InMem → Redis → S3/GCS/LocalFileCache), supporting per-level TTL and automatic promotion of hot data. [1] [2] [3]S3Cache(AWS),GCSCache(Google Cloud), andLocalFileCache(filesystem) as new object storage backends with features like TTL via metadata, optional compression, and dedupe to avoid redundant writes. [1] [2] [3]dedupe_writesfor RedisCache, S3Cache, GCSCache, and LocalFileCache to skip rewriting unchanged payloads, reducing storage and network costs. [1] [2]Documentation and Usage Guides
README.mdto document new backends,ChainCache, dedupe semantics, and advanced configuration, including code samples and comparison tables. [1] [2] [3] [4]BGCache(background refresh, single-writer/multi-reader) with best practices for error handling, cache topologies, and integration with object storage.Testing and CI Infrastructure
.github/workflows/ci.yml) to run unit tests and integration tests for S3 (via MinIO) and GCS (via fake-gcs-server) backends, ensuring reliability of new storage layers.docker-compose.ymlfor local development/testing with MinIO and fake-gcs-server services.Changelog and Maintenance
CHANGELOG.mdto reflect all new features, fixes, and documentation improvements for version 0.2.2. [1] [2]These changes significantly improve the flexibility, scalability, and production readiness of the library, especially for teams needing robust multi-tier caching and object storage integration.