Skip to content

docs: update guides to match dev setup guide #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
swarna1101 merged 1 commit into from
Dec 1, 2025
Merged

docs: update guides to match dev setup guide #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/guides/01-getting-started-cli.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Getting Started with mump2p CLI

The `mump2p` CLI is the quickest way to interact with [Optimum Network](https://github.com/getoptimum/optimum-p2p) without running your own infrastructure.
The `mump2p` CLI is the quickest way to interact with Optimum Network without running your own infrastructure.

In the next 5 minutes, you'll have:

Expand Down
44 changes: 27 additions & 17 deletions docs/guides/02-getting-started-docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ Choose the deployment mode that best fits your use case:

| Component | Purpose | Docker Images |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
| **mump2p Node** | RLNC-enabled mesh peer, encodes/decodes message shards, handles peer discovery and subscriptions. Optional gRPC API for direct clients. | `getoptimum/p2pnode:v0.0.1-rc2` |
| **Optimum Proxy** | Bridges clients and the mesh, manages subscriptions, shard reassembly, threshold logic, and node selection. | `getoptimum/proxy:v0.0.1-rc3` |
| **mump2p Node** | RLNC-enabled mesh peer, encodes/decodes message shards, handles peer discovery and subscriptions. Optional gRPC API for direct clients. | `getoptimum/p2pnode:${P2P_NODE_VERSION-latest}` |
| **Optimum Proxy** | Bridges clients and the mesh, manages subscriptions, shard reassembly, threshold logic, and node selection. | `getoptimum/proxy:${PROXY_VERSION-latest}` |



Expand Down Expand Up @@ -96,13 +96,21 @@ Before starting, create your `.env` file:
cp .env.example .env
```

**Important:** After copying, you need to replace the `BOOTSTRAP_PEER_ID` in your `.env` file with the peer ID generated by `make generate-identity`.

**Workflow:**

1. Run `make generate-identity` - this creates a unique peer ID
2. Copy the generated peer ID from the output
3. Edit your `.env` file and replace the example `BOOTSTRAP_PEER_ID` with your generated one

Edit with your values:

```bash
BOOTSTRAP_PEER_ID=<your-generated-peer-id>
CLUSTER_ID=my-cluster
PROXY_VERSION=v0.0.1-rc7
P2P_NODE_VERSION=v0.0.1-rc6
PROXY_VERSION=v0.0.1-rc16
P2P_NODE_VERSION=v0.0.1-rc16
```

> **Complete Guide:** [Environment configuration](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#environment-variables-env)
Expand All @@ -117,7 +125,7 @@ make generate-identity

This creates `./identity/p2p.key` with your unique Peer ID.

> **Complete Guide:** [Identity generation and Makefile commands](https://github.com/getoptimum/optimum-dev-setup-guide#quick-start) - all make commands, direct binary usage
> **Complete Guide:** [Identity generation and Makefile commands](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#getting-started) - all make commands, direct binary usage

## 5. Mode A — OptimumProxy + mump2p (Recommended)

Expand Down Expand Up @@ -156,20 +164,20 @@ services:
> **Complete Docker Compose:**
>
> * [Full configuration with 2 proxies + 4 nodes](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docker-compose-optimum.yml)
> * [All environment variables](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#p2p-node-variables)
> * [All environment variables](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#environment-variables-env)

### Start the Network

```sh
export BOOTSTRAP_PEER_ID=<your-peer-id>
docker compose up -d
docker-compose -f docker-compose-optimum.yml up --build -d
```

### Verify Health

```sh
# Check containers
docker compose ps
docker-compose -f docker-compose-optimum.yml ps

# Test endpoints
curl http://localhost:8081/api/v1/health # Proxy
Expand Down Expand Up @@ -251,12 +259,14 @@ services:
```

> **Complete Docker Compose:** [Full configuration for direct P2P mode](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docker-compose-optimum.yml)
>
> **Note:** The docker-compose file uses environment variables from `.env` for versions. Ensure your `.env` file has `PROXY_VERSION` and `P2P_NODE_VERSION` set.

### Start and Verify

```sh
export BOOTSTRAP_PEER_ID=<your-peer-id>
docker compose up -d
docker-compose -f docker-compose-optimum.yml up --build -d
curl http://localhost:9091/api/v1/health
```

Expand All @@ -266,15 +276,15 @@ Connect using the P2P client with trace handling and metrics:

```bash
# Subscribe
./p2p-client -mode=subscribe -topic=testtopic --addr=127.0.0.1:33221
./grpc_p2p_client/p2p-client -mode=subscribe -topic=testtopic --addr=127.0.0.1:33221

# Publish
./p2p-client -mode=publish -topic=testtopic -msg="Hello" --addr=127.0.0.1:33222
./grpc_p2p_client/p2p-client -mode=publish -topic=testtopic -msg="Hello" --addr=127.0.0.1:33222
```

> **Complete Implementation:**
>
> * [P2P Client Source](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/p2p_client.go)
> * [P2P Client Source](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/cmd/single/main.go)
> * [Usage Guide with Makefile commands](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#using-p2p-nodes-directly-optional--no-proxy)
> * [Message format explanation](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#understanding-message-output-format)

Expand All @@ -284,16 +294,16 @@ For all configuration variables, see the [Parameters Section](./03-parameters.md

### "Connection refused" from client

* Ensure you’re pointing to the host-mapped ports (e.g., 33221, 8081).
* Run docker compose ps and confirm port bindings.
* Ensure you're pointing to the host-mapped ports (e.g., 33221, 8081).
* Run `docker-compose -f docker-compose-optimum.yml ps` and confirm port bindings.
* Firewalls: allow inbound localhost traffic.

### Proxy can’t reach nodes

* Inside the proxy container, resolve and ping node hosts:

```sh
docker compose exec proxy sh -lc 'nc -zv p2pnode-1 33212'
docker-compose -f docker-compose-optimum.yml exec proxy-1 sh -lc 'nc -zv p2pnode-1 33212'
```

* Make sure `P2P_NODES` hostnames match the `service names` in compose.
Expand All @@ -311,11 +321,11 @@ docker compose exec proxy sh -lc 'nc -zv p2pnode-1 33212'
Stop:

```sh
docker compose down
docker-compose -f docker-compose-optimum.yml down
```

Full reset (containers, volumes, images created by this compose file):

```sh
docker compose down -v --rmi local
docker-compose -f docker-compose-optimum.yml down -v --rmi local
```
28 changes: 13 additions & 15 deletions docs/guides/03-parameters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,17 +111,17 @@ Example Docker service:
proxy-1:
image: 'getoptimum/proxy:${PROXY_VERSION-latest}'
environment:
- PROXY_HTTP_PORT=:8080
- PROXY_PORT=:8080
- PROXY_GRPC_PORT=:50051
- CLUSTER_ID=proxy-1
- CLUSTER_ID=${CLUSTER_ID}
- ENABLE_AUTH=false
- LOG_LEVEL=debug
- P2P_NODES=p2pnode-1:33212,p2pnode-2:33212
```

| Parameter | Default | Purpose |
| --------------------------------- | ------- | ---------------------------------------------------------- |
| `PROXY_HTTP_PORT` | :8080 | HTTP API port for clients. |
| `PROXY_PORT` | :8080 | HTTP API port for clients. |
| `PROXY_GRPC_PORT` | :50051 | gRPC API port for clients. |
| `ENABLE_AUTH` | false | Enable Auth0 authentication. |
| `AUTH0_DOMAIN` / `AUTH0_AUDIENCE` | (none) | Auth0 settings (required if `ENABLE_AUTH`=true). |
Expand All @@ -139,16 +139,15 @@ The following table shows the **production defaults**, which are optimized for t
```yaml
environment:
- NODE_MODE=optimum
- LOG_LEVEL=production
- LOG_LEVEL=debug
- CLUSTER_ID=my-cluster
- SIDECAR_PORT=33212
- API_PORT=8081
- API_PORT=9090
- IDENTITY_DIR=/identity
- OPTIMUM_PORT=7070
- OPTIMUM_MAX_MSG_SIZE=1048576
- OPTIMUM_RANDOM_MSG_SIZE=512
- OPTIMUM_MESH_TARGET=6
- OPTIMUM_MESH_MIN=4
- OPTIMUM_MESH_MIN=3
- OPTIMUM_MESH_MAX=12
- OPTIMUM_SHARD_FACTOR=4
- OPTIMUM_SHARD_MULT=1.5
Expand All @@ -161,10 +160,10 @@ environment:
```yaml
environment:
- NODE_MODE=gossipsub
- LOG_LEVEL=production
- LOG_LEVEL=debug
- CLUSTER_ID=my-cluster
- SIDECAR_PORT=33212
- API_PORT=8081
- API_PORT=9090
- IDENTITY_DIR=/identity
- GOSSIPSUB_PORT=6060
- GOSSIPSUB_MAX_MSG_SIZE=1048576
Expand All @@ -178,17 +177,16 @@ environment:

| Parameter | Default Value | Used In Mode | Description |
| ---------------------------- | ------------- | ------------ | ---------------------------------------------- |
| `LOG_LEVEL` | production | Both | Log verbosity (production/debug/info/warn/error) |
| `LOG_LEVEL` | debug | Both | Log verbosity (debug/info/warn/error) |
| `CLUSTER_ID` | "" | Both | Logical group name for metrics |
| `NODE_MODE` | "" | Both | Protocol mode (optimum/gossipsub) |
| `SIDECAR_PORT` | 33212 | Both | gRPC sidecar port |
| `API_PORT` | 8081 | Both | HTTP API port |
| `IDENTITY_DIR` | /tmp | Both | Directory for node private key |
| `API_PORT` | 9090 | Both | HTTP API port |
| `IDENTITY_DIR` | /identity | Both | Directory for node private key |
| `OPTIMUM_PORT` | 7070 | optimum | TCP port for RLNC gossip |
| `OPTIMUM_MAX_MSG_SIZE` | 1048576 | optimum | Max message size (1MB) |
| `OPTIMUM_RANDOM_MSG_SIZE` | 512 | optimum | Random message size for testing |
| `OPTIMUM_MESH_TARGET` | 6 | optimum | Desired peers in mesh |
| `OPTIMUM_MESH_MIN` | 4 | optimum | Minimum peers before adding more |
| `OPTIMUM_MESH_MIN` | 3 | optimum | Minimum peers before adding more |
| `OPTIMUM_MESH_MAX` | 12 | optimum | Max peers before pruning |
| `OPTIMUM_SHARD_FACTOR` | 4 | optimum | Number of shards per message |
| `OPTIMUM_SHARD_MULT` | 1.5 | optimum | Redundancy multiplier (extra shards) |
Expand All @@ -198,7 +196,7 @@ environment:
| `GOSSIPSUB_MESH_TARGET` | 6 | gossipsub | Desired peers in mesh |
| `GOSSIPSUB_MESH_MIN` | 4 | gossipsub | Minimum peers before adding more |
| `GOSSIPSUB_MESH_MAX` | 12 | gossipsub | Max peers before pruning |
| `BOOTSTRAP_PEERS` | [] | Both | List of peer multiaddrs for bootstrap |
| `BOOTSTRAP_PEERS` | (none) | Both | List of peer multiaddrs for bootstrap |

> **Note:** These are the production defaults used by mump2p nodes. For experimental tuning, see [Common Experiments](./04-experiments.md).

30 changes: 16 additions & 14 deletions docs/guides/04-experiments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Each experiment below lists the **goal**, a quick **how-to**, and **what to obse
You can run them using:

* **mump2p CLI** (see [CLI Guide](./01-getting-started-cli.md))
* **gRPC client** (with `MessageTraceGossipSub` or `MessageTraceOptimump2p` for protocol metrics)
* **gRPC P2P client** (trace collection is automatic when subscribing - see [Trace Collection](#metrics-collection))



Expand Down Expand Up @@ -112,33 +112,35 @@ You can run them using:
**Expected Result:** mump2p should handle higher stress levels before failing compared to GossipSub.


> **Tip:** Enable protocol traces in the gRPC client to get hop-by-hop delivery info:
>
> * `MessageTraceGossipSub` for GossipSub mode.
> * `MessageTraceOptimump2p` for mump2p mode.
> **Tip:** Trace collection is automatic when using the gRPC P2P client. Simply subscribe to a topic and trace events will be automatically parsed and displayed. See [Trace Collection](#metrics-collection) for details.

## Metrics Collection

For comprehensive metrics collection during experiments, use the gRPC P2P client with trace handling:

**[P2P Client with Metrics Collection](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#collecting-trace-data-for-experiments)**

**[Complete Code](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/p2p_client.go)**
**[Complete Code](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/cmd/single/main.go)**

The client automatically captures and displays:
The client automatically captures and displays trace events when you subscribe:

* **GossipSub traces**: Peer routing, message delivery status, hop counts, latency
* **mump2p traces**: Shard encoding/decoding, reconstruction efficiency, redundancy metrics
* **Message-level data**: Delivery success rates, end-to-end latency, bandwidth usage

**Key trace handlers:**
**Usage:**

```bash
# Subscribe to a topic - trace events are automatically collected and displayed
./grpc_p2p_client/p2p-client -mode=subscribe -topic=your-topic --addr=127.0.0.1:33221
```

Trace parsing is implemented in `grpc_p2p_client/shared/utils.go` and handles both `ResponseType_MessageTraceGossipSub` and `ResponseType_MessageTraceMumP2P` automatically.

```go
case protobuf.ResponseType_MessageTraceGossipSub:
fmt.Printf("[TRACE] GossipSub trace received: %s\n", string(resp.GetData()))
For multi-node experiments with trace data collection, use the multi-subscribe client:

case protobuf.ResponseType_MessageTraceOptimump2p:
fmt.Printf("[TRACE] mump2p trace received: %s\n", string(resp.GetData()))
```bash
./grpc_p2p_client/p2p-multi-subscribe -topic=test-topic -ipfile=ips.txt -output-data=data.tsv -output-trace=trace.tsv
```

Use this client instead of the CLI for detailed performance analysis during experiments.
See the [dev setup guide](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#multi-node-client-tools) for more details on multi-node client tools.
35 changes: 22 additions & 13 deletions docs/guides/05-faq-glossary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,24 +20,28 @@ It includes detailed troubleshooting for:

### Q: `docker run ... generate-key` command doesn't work

**A:** Use the identity generation script instead:
**A:** Use the identity generation script or Makefile command:

```bash
curl -sSL https://raw.githubusercontent.com/getoptimum/optimum-dev-setup-guide/main/script/generate-identity.sh | bash
# Using Makefile (recommended)
make generate-identity

# Or using the script directly
./script/generate-identity.sh
```

This generates `p2p.key` and exports `BOOTSTRAP_PEER_ID` automatically.
This generates `./identity/p2p.key` and displays the `BOOTSTRAP_PEER_ID` that you need to add to your `.env` file.

### Container Startup Issues

### Q: Containers fail to start or can't connect to each other

**A:** Common fixes:

1. **Check Docker images**: Use correct versions (`getoptimum/proxy:v0.0.1-rc3`, `getoptimum/p2pnode:v0.0.1-rc2`)
1. **Check Docker images**: Use correct versions from your `.env` file (`PROXY_VERSION=v0.0.1-rc16`, `P2P_NODE_VERSION=v0.0.1-rc16` by default)
2. **Network conflicts**: Change subnet in docker-compose if `172.28.0.0/16` conflicts
3. **Port conflicts**: Ensure ports 8080, 8081, 33221, 9091, 7071 are available
4. **Platform issues**: Add `platform: linux/amd64` for M1 Macs
3. **Port conflicts**: Ensure ports 8081, 8082, 50051, 50052, 33221-33224, 9091-9094, 7071-7074 are available
4. **Platform issues**: Add `platform: linux/amd64` for M1 Macs (already included in docker-compose-optimum.yml)

### Q: "Connection refused" when clients try to connect

Expand All @@ -60,14 +64,16 @@ This generates `p2p.key` and exports `BOOTSTRAP_PEER_ID` automatically.
3. **Use latest client examples**: Reference [`optimum-dev-setup-guide/docs/guide.md#grpc-proxy-client-implementation`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#grpc-proxy-client-implementation)

**[Complete Code](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_proxy_client/proxy_client.go)**

For P2P direct client, see [`grpc_p2p_client/cmd/single/main.go`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/cmd/single/main.go)

### Q: Getting "method not found" or protobuf errors

**A:** Use the correct protobuf definitions from [`optimum-dev-setup-guide/docs/guide.md`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#api-reference):

* See the [API Reference section](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#api-reference) for complete protobuf definitions
* All proto files are available in the repository's `grpc_*_client/proto/` directories:
* [`grpc_proxy_client/proto/gateway_stream.proto`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_proxy_client/proto/gateway_stream.proto)
* [`grpc_proxy_client/proto/proxy_stream.proto`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_proxy_client/proto/proxy_stream.proto)
* [`grpc_p2p_client/proto/p2p_stream.proto`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/proto/p2p_stream.proto)


Expand Down Expand Up @@ -97,14 +103,17 @@ This generates `p2p.key` and exports `BOOTSTRAP_PEER_ID` automatically.

When something doesn't work:

1. **Check container logs**: `docker logs <container-name>`
2. **Verify network connectivity**: `docker network ls` and `docker network inspect`
3. **Test basic connectivity**: `curl http://localhost:8080/health`
4. **Check authentication**: `mump2p whoami`
5. **Verify versions**: Use latest CLI and Docker images
1. **Check container logs**: `docker-compose -f docker-compose-optimum.yml logs <service-name>` or `docker logs <container-name>`
2. **Verify network connectivity**: `docker network ls` and `docker network inspect optimum-dev-setup-guide_optimum-network`
3. **Test basic connectivity**:
* Proxy: `curl http://localhost:8081/api/v1/health`
* P2P Node: `curl http://localhost:9091/api/v1/health`
4. **Check authentication**: `mump2p whoami` (if using CLI)
5. **Verify versions**: Check your `.env` file for `PROXY_VERSION` and `P2P_NODE_VERSION` (default: v0.0.1-rc16)
6. **Check service status**: `docker-compose -f docker-compose-optimum.yml ps`

### Getting Help

* **CLI Issues**: [mump2p-cli FAQ](https://github.com/getoptimum/mump2p-cli#faq---common-issues--troubleshooting)
* **Setup Issues**: Check [`optimum-dev-setup-guide/docs/guide.md`](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md)
* **Protocol Questions**: See [mump2p Documentation](./p2p.md)
* **Protocol Questions**: See [mump2p Documentation](../learn/overview/p2p.md)