KeeVee is a distributed key-value storage system implemented in Rust, featuring Paxos consensus for strong consistency and data replication. It is designed to demonstrate the core principles of distributed systems, including leader election, log replication, and fault tolerance.
- Distributed Consensus: Implements the Paxos algorithm to ensure all nodes agree on the sequence of state changes.
- Strong Consistency: Guarantees that confirmed writes are replicated to a majority of nodes.
- Automatic Leader Election: Nodes automatically elect a leader to coordinate updates. If the leader fails, a new one is elected.
- Request Forwarding: Write requests sent to follower nodes are automatically forwarded to the current leader.
- Persistent Storage: Uses RocksDB for reliable, on-disk data storage.
- HTTP API: Provides a simple RESTful interface for interacting with the key-value store.
- Cluster Management: Includes scripts to easily spin up and test a local 3-node cluster.
- Language: Rust (Edition 2021)
- Runtime: Tokio (Async I/O)
- API Server: Axum
- Storage Engine: RocksDB
- Consensus: Custom Paxos implementation
- Communication: HTTP/JSON for both client-server and inter-node (RPC) communication.
- Rust & Cargo: Ensure you have the latest stable version of Rust installed.
- Unix-like Environment: The provided scripts are written in Bash (tested on macOS).
To build the project, run:
cargo buildYou can start a 3-node cluster on your local machine using the provided helper script. This will build the project and launch 3 nodes listening on ports 8001, 8002, and 8003.
./scripts/start_server.shLogs and data for each node will be stored in the ./tmp/data directory.
To stop the cluster, you can use pkill -f "target/debug/keevee" or simply kill the script if it's running in the foreground (though the script launches nodes in the background).
The cluster exposes a simple HTTP API. You can send requests to any node (e.g., http://127.0.0.1:8001).
Set a Value (PUT)
curl -X PUT http://127.0.0.1:8001/kv/<key>/<value>Get a Value (GET)
curl http://127.0.0.1:8001/kv/<key>Delete a Value (DELETE)
curl -X DELETE http://127.0.0.1:8001/kv/<key>Check Node Status
curl http://127.0.0.1:8001/is_leaderThe project includes several scripts to verify functionality and performance.
Tests basic Put, Get, and Delete operations on the cluster.
./scripts/test_CRUD.shPerforms random writes to different nodes (leaders and followers) and verifies that the data is correctly replicated to all nodes in the cluster. This confirms that request forwarding and consensus are working.
./scripts/test_cross_nodes.shMeasures the performance of the cluster by performing a series of operations on each node and calculating the execution time.
./scripts/test_benchmarks.shsrc/main.rs: Entry point and CLI argument parsing.src/node.rs: Core node logic, HTTP server, and heartbeat mechanism.src/paxos/: Implementation of the Paxos consensus algorithm.src/storage.rs: Abstraction over the RocksDB storage engine.src/log.rs: Replicated log implementation.src/statemachine.rs: State machine that applies committed log entries to storage.
Keevee ensures strong consistency for all write operations using the Paxos consensus algorithm.
- Leader-Based Writes: All write requests (PUT/DELETE) are handled by the cluster leader. If a client sends a write request to a follower node, it is automatically forwarded to the leader.
- Consensus: Before a write is acknowledged, the leader replicates the proposed value to a majority of nodes (quorum) in the cluster.
- Commit: Once a majority has accepted the value, the leader commits the entry to its log and instructs followers to do the same.
- Durability: Committed entries are applied to the state machine and persisted to disk using RocksDB, ensuring data survives node restarts.
The following diagram illustrates the flow of a write operation when a client connects to a follower node:
sequenceDiagram
participant Client
participant Follower
participant Leader
participant OtherFollower
Client->>Follower: PUT /kv/key/value
Note over Follower: Not Leader
Follower->>Leader: Forward Request
Note over Leader: Propose Value (Paxos)
Leader->>Follower: Accept Request
Leader->>OtherFollower: Accept Request
Follower-->>Leader: Accepted
OtherFollower-->>Leader: Accepted
Note over Leader: Majority Reached
Leader->>Leader: Commit & Apply
Leader->>Follower: Commit Message
Leader->>OtherFollower: Commit Message
Leader-->>Follower: Response (Success)
Follower-->>Client: Response (Success)
| Method | Endpoint | Description |
|---|---|---|
GET |
/kv/:key |
Retrieve the value for a specific key. |
PUT |
/kv/:key/:value |
Set a value for a specific key. |
DELETE |
/kv/:key |
Delete a key-value pair. |
| Method | Endpoint | Description |
|---|---|---|
GET |
/v1/cluster/status |
Get the current status of the node (Role, Leader ID, Log Index). |
GET |
/v1/cluster/nodes |
List all known nodes in the cluster. |
GET |
/is_leader |
Simple check to see if the current node is the leader. |
POST |
/heartbeat |
Internal endpoint for leader heartbeats. |
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.