Skip to content

Conversation

@copyleftdev
Copy link
Owner

Completes ClickHouse storage with real integration tests using testcontainers.

Components:

  • ClickHouseClient with connection pooling
  • SessionWriter (single + batch writes)
  • Schema initialization with indexes and TTL
  • Testcontainers integration tests

Tests (5 integration tests):

  • Connection and health checks
  • Schema creation
  • Single session write
  • Batch session writes (10 sessions)
  • Query by fingerprint

Run integration tests (requires Docker):

cargo test --package scrybe-storage -- --ignored

Schema features:

  • Monthly partitioning
  • 90-day automatic TTL
  • Bloom filter on fingerprint_hash
  • Token bloom filter on IP

Closes #5

…ners (100%)

Implements full ClickHouse storage layer with real integration tests.

Components:
- ClickHouseClient with connection pooling and health checks
- SessionWriter with single and batch write support
- Schema initialization with indexes and 90-day TTL
- Session row serialization to ClickHouse format

Testing:
- Testcontainers for real ClickHouse integration tests
- 5 comprehensive integration tests (Docker required)
- Tests: connection, schema, single write, batch write, queries

Run integration tests:
  cargo test --package scrybe-storage -- --ignored

Schema features:
- Monthly partitioning for efficient queries
- 90-day TTL for automatic cleanup
- Bloom filter on fingerprint_hash
- Token bloom filter on IP addresses

This delivers 100% of RFC-0005 with production-quality testing.

Closes #5
Refs: RFC-0005
@copyleftdev copyleftdev merged commit fe9a288 into main Nov 23, 2025
8 checks passed
@copyleftdev copyleftdev deleted the feature/issue-5-clickhouse-storage branch November 23, 2025 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC-0005: Implement ClickHouse Storage Schema

2 participants