TFGrid Compose

Universal deployment orchestrator for ThreeFold Grid applications.

Overview

TFGrid Compose is a production-ready deployment platform that makes deploying applications on ThreeFold Grid as simple as tfgrid-compose up.

Key Features:

✅ One-command deployment - From zero to running app in 2-3 minutes
✅ Context file support - Set app once, use short commands everywhere
✅ App-specific commands - Apps define custom commands (tfgrid-compose create, tfgrid-compose run)
✅ Auto-install - Automatically sets up PATH in your shell
✅ Multiple patterns - single-vm, gateway, k3s (coming soon)
✅ Smart validation - Checks prerequisites and prevents errors
✅ Remote execution - Run commands on deployed VMs from your local machine
✅ State management - Tracks all deployments automatically
✅ Idempotency - Safe to retry, prevents duplicate deployments

📚 Documentation

Complete documentation: docs.tfgrid.studio

Quick Links

Getting Started - Installation and first deployment
Architecture - System design and components
Troubleshooting - Common errors and solutions
AI Agent Guide - Deploy AI coding assistant
Pattern Contract - Create custom patterns

Community

Contributing - How to contribute
Code of Conduct - Community guidelines
Security Policy - Report vulnerabilities

Structure

tfgrid-compose/
├── cli/                     # tfgrid-compose CLI tool
├── core/                    # Core orchestration logic
├── patterns/                # Deployment patterns
│   └── single-vm/           # Single VM pattern (MVP)
├── lib/                     # Shared modules and roles
└── docs/                    # Documentation

Current Status

🎉 v0.13.4 - Production Ready

All 3 core deployment patterns are production-ready!

v1.0 Features ✅

All 3 Core Patterns Complete! ✅

Single-VM pattern - Development, databases, AI agents
Gateway pattern - Production web apps with public IPv4, SSL, load balancing
K3s pattern - Kubernetes clusters for cloud-native apps

Featured Applications (6 Production-Ready)

tfgrid-wordpress - WordPress + Caddy + MariaDB with DNS automation
tfgrid-nextcloud - Nextcloud All-in-One cloud platform
tfgrid-erpnext - ERPNext business ERP system
tfgrid-ai-agent - AI coding assistant
tfgrid-ai-stack - Complete AI development environment with Git hosting
tfgrid-gitea - Self-hosted Git service

New Features ✨

Interactive mode (-i flag) - Guided deployment with prompts
DNS automation - Cloudflare, Name.com, Namecheap support
Local dashboard - Visual deployment management

Coming Soon 🚧

Automated test suite
Shell completion (bash/zsh/fish)
Video tutorials

Patterns

single-vm ✅ Production Ready

Single VM deployment with Wireguard and Mycelium networking.

Use cases:

Development environments
Databases (PostgreSQL, MongoDB, Redis)
Internal services
AI agents
Background workers

Documentation: patterns/single-vm/

gateway ✅ Production Ready

Gateway VM with public IPv4 + backend VMs with private networking.

Use cases:

Production web applications
E-commerce sites
Multi-tier applications
Public-facing services

Features:

Public IPv4 on gateway
Nginx/HAProxy reverse proxy
Free SSL/TLS (Let's Encrypt)
Load balancing & health checks
Private backend network
Network redundancy (WireGuard + Mycelium)

Documentation: patterns/gateway/README.md

k3s ✅ Production Ready

Kubernetes cluster (K3s) with control plane, workers, and management node.

Use cases:

Cloud-native applications
Microservices architectures
Production SaaS platforms
High availability requirements

Features:

Lightweight Kubernetes (K3s)
MetalLB load balancer
Nginx Ingress Controller
Management node with kubectl, helm, k9s
Local-path storage provisioner
HA control plane support

Documentation: patterns/k3s/README.md

Fresh Machine Setup (Complete Prerequisites)

If you're starting on a new machine without any prerequisites, follow this complete setup guide.

Prerequisites Checklist

Tool	Required	Purpose
OpenTofu (or Terraform)	✅ Yes	Infrastructure provisioning
Ansible	✅ Yes	Configuration management
WireGuard	✅ Yes	Private networking
Mycelium	✅ Yes	ThreeFold overlay network
yq (Mike Farah's Go version)	✅ Yes	YAML parsing
ThreeFold mnemonic	✅ Yes	Grid authentication

Ubuntu/Debian Complete Setup

# 1. OpenTofu (recommended over Terraform - open source)
curl -fsSL https://get.opentofu.org/install-opentofu.sh | sudo bash -s -- --install-method deb

# 2. Ansible
sudo apt update && sudo apt install -y ansible

# 3. WireGuard
sudo apt install -y wireguard

# 4. yq (Mike Farah's Go version - NOT the Python jq-wrapper!)
sudo wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq
sudo chmod +x /usr/bin/yq

# 5. Mycelium (ThreeFold overlay network)
wget https://github.com/threefoldtech/mycelium/releases/latest/download/mycelium-x86_64-unknown-linux-musl.tar.gz
tar -xzf mycelium-x86_64-unknown-linux-musl.tar.gz
sudo mv mycelium /usr/local/bin/
sudo chmod +x /usr/local/bin/mycelium
rm mycelium-x86_64-unknown-linux-musl.tar.gz

# 6. Setup Mycelium as a service (with all public peers)
sudo tee /etc/systemd/system/mycelium.service > /dev/null << 'EOF'
[Unit]
Description=Mycelium Network
After=network.target

[Service]
ExecStart=/usr/local/bin/mycelium \
  --peers tcp://188.40.132.242:9651 \
  --peers tcp://136.243.47.186:9651 \
  --peers tcp://185.69.166.7:9651 \
  --peers tcp://185.69.166.8:9651 \
  --peers tcp://65.21.231.58:9651 \
  --peers tcp://65.109.18.113:9651 \
  --peers tcp://209.159.146.190:9651 \
  --peers tcp://5.78.122.16:9651 \
  --peers tcp://5.223.43.251:9651 \
  --peers tcp://142.93.217.194:9651
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable mycelium
sudo systemctl start mycelium

# 7. Verify all installations
echo "=== Verifying installations ==="
tofu --version || terraform --version
ansible --version | head -1
wg --version
mycelium --version
yq --version  # Should show: yq (https://github.com/mikefarah/yq/) version v4.x.x

macOS Complete Setup

# 1. OpenTofu
brew install opentofu

# 2. Ansible
brew install ansible

# 3. WireGuard
brew install wireguard-tools

# 4. yq (Mike Farah's version)
brew install yq

# 5. Mycelium
brew install threefoldtech/tap/mycelium
# Start with: mycelium --peers tcp://188.40.132.242:9651 (see peers list above)

Configure ThreeFold Credentials

# Interactive signin (recommended)
tfgrid-compose signin

# Verify credentials
tfgrid-compose signin --check

That's it! The mnemonic is stored in ~/.config/tfgrid-compose/credentials.yaml and tfgrid-compose reads it automatically.

Verify Prerequisites

# Run tfgrid-compose - it will check all prerequisites
tfgrid-compose up --help

Common Issues

Issue	Symptom	Fix
Wrong yq installed	`yq --version` shows `0.0.0` or mentions Python	Uninstall Python yq, install Mike Farah's version
Mnemonic not found	"ThreeFold mnemonic not configured"	Run `tfgrid-compose signin`
Pattern not detected	"No pattern specified" error	Install correct yq, or use `--pattern single-vm`
Mycelium not connected	Mycelium connectivity test fails	Check `sudo systemctl status mycelium`
Node out of space	"not enough space left in pools"	Try different node or reduce `--disk` size

Quick Start

1. Install tfgrid-compose

# Setup standard workspace
mkdir -p ~/code/github.com/tfgrid-studio
cd ~/code/github.com/tfgrid-studio

# Clone repositories
git clone https://github.com/tfgrid-studio/tfgrid-compose
git clone https://github.com/tfgrid-studio/tfgrid-ai-agent

# Install (auto-configures PATH)
cd tfgrid-compose
make install

This automatically adds tfgrid-compose to your PATH and creates a default tfgrid shortcut!

Shortcuts: Use tfgrid instead of tfgrid-compose for shorter commands, or create your own with tfgrid-compose shortcut <name>.

Note: We use ~/code/github.com/{org}/{repo} as the standard workspace structure. See WORKSPACE_STANDARD.md for details.

See Quick Start Guide for detailed setup.

2. Configure

# Store your mnemonic
mkdir -p ~/.config/threefold
echo "your twelve word mnemonic" > ~/.config/threefold/mnemonic
chmod 600 ~/.config/threefold/mnemonic

# Create context file (optional but recommended)
echo "app: ../tfgrid-ai-agent" > .tfgrid-compose.yaml

3. Deploy & Use

# Deploy application (uses context)
tfgrid-compose up

# Use AI agent with simple commands
tfgrid-compose projects
tfgrid-compose create
tfgrid-compose run my-project

# Specify app path every time
tfgrid-compose up ../tfgrid-ai-agent
tfgrid-compose status ../tfgrid-ai-agent
tfgrid-compose ssh ../tfgrid-ai-agent
tfgrid-compose down ../tfgrid-ai-agent

Basic Commands

# Deploy
tfgrid-compose up [app-path]

# Execute commands on VM
tfgrid-compose exec [app-path] <command>

# AI Agent management (requires context)
tfgrid-compose projects
tfgrid-compose create
tfgrid-compose run <project>
tfgrid-compose monitor <project>
tfgrid-compose stop <project>

# Status & logs
tfgrid-compose status [app-path]
tfgrid-compose ssh [app-path]
tfgrid-compose logs [app-path]

# Destroy
tfgrid-compose down [app-path]

# Shortcuts (optional)
tfgrid-compose shortcut <name>    # Create custom shortcut
tfgrid-compose shortcut --list    # List shortcuts

Command Shortcuts

Save typing with custom shortcuts:

# Default shortcut (created during install)
tfgrid up              # Instead of: tfgrid-compose up

# Interactive mode - easy menu-driven interface
tfgrid-compose shortcut
# 1) Create a new shortcut
# 2) Remove a shortcut
# 3) List all shortcuts
# 4) Reset to default (tfgrid)
# 5) Exit

# Or create directly
tfgrid-compose shortcut tf
tf up                  # Even shorter!

# List all shortcuts
tfgrid-compose shortcut --list

# Remove a shortcut
tfgrid-compose shortcut --remove tf

Complete Workflow Example

# 1. Setup context
echo "app: ../tfgrid-ai-agent" > .tfgrid-compose.yaml

# 2. Deploy
tfgrid-compose up

# 3. Use AI agent
tfgrid-compose create
# Follow prompts: name, duration, prompt

# 4. Monitor
tfgrid-compose projects
tfgrid-compose monitor my-project

# 5. Cleanup
tfgrid-compose stop my-project
tfgrid-compose down

How It Works

Complete deployment flow:

✅ Validates prerequisites and configuration
✅ Plans infrastructure with Terraform
✅ Provisions VM on ThreeFold Grid
✅ Configures WireGuard networking
✅ Waits for SSH to be ready
✅ Runs Ansible playbooks
✅ Deploys application source code
✅ Runs deployment hooks (setup → configure → healthcheck)
✅ Verifies deployment success
✅ Saves state for management

Total time: 2-3 minutes ⚡

Documentation

Getting Started

Quick Start Guide - Get started in 5 minutes
AI Agent Guide - Complete AI agent integration guide
Context File Usage - Using .tfgrid-compose.yaml for simpler commands

Development

TODO - Roadmap and future features
Implementation Summary - Development notes
Contributing - How to contribute

Patterns

See full pattern documentation above or run:

tfgrid-compose patterns

Development

Running Tests

# Validation tests
./tests/test-validation.sh

# Full lifecycle test
make up APP=../tfgrid-ai-agent
make status APP=../tfgrid-ai-agent
make down APP=../tfgrid-ai-agent

Troubleshooting

Full guide: docs.tfgrid.studio/troubleshooting/guide

Quick fixes:

Missing prerequisites: tfgrid-compose up will tell you what's missing
Existing deployment: tfgrid-compose down <app> then redeploy
State corruption: rm -rf .tfgrid-compose/ (⚠️ removes state)

Contributing

We welcome contributions! Please see our Contributing Guide and Code of Conduct.

Quick start:

Fork the repository
Create a feature branch
Make your changes
Submit a pull request

Security issues: See our Security Policy

License

Apache 2.0 - See LICENSE file

Part of: TFGrid Studio
Status: ✅ Complete First Layer
Version: 0.13.4
Patterns: 3/3 Production Ready

All deployment patterns complete and ready for production! 🚀

Support

📚 Documentation: docs.tfgrid.studio
🐛 Issues: GitHub Issues
💬 Discussions: GitHub Discussions
📧 Contact: tfgrid.studio/contact

Links

🌐 Main Site: tfgrid.studio
📝 Changelog: CHANGELOG.md

Name		Name	Last commit message	Last commit date
Latest commit History 534 Commits
cli		cli
core		core
dashboard		dashboard
docs		docs
examples		examples
patterns		patterns
scripts		scripts
tests		tests
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
SECURITY.md		SECURITY.md
VERSION		VERSION
install.sh		install.sh

License

tfgrid-studio/tfgrid-compose

Folders and files

Latest commit

History

Repository files navigation

TFGrid Compose

Overview

📚 Documentation

Quick Links

Community

Structure

Current Status

v1.0 Features ✅

All 3 Core Patterns Complete! ✅

Featured Applications (6 Production-Ready)

New Features ✨

Coming Soon 🚧

Patterns

single-vm ✅ Production Ready

gateway ✅ Production Ready

k3s ✅ Production Ready

Fresh Machine Setup (Complete Prerequisites)

Prerequisites Checklist

Ubuntu/Debian Complete Setup

macOS Complete Setup

Configure ThreeFold Credentials

Verify Prerequisites

Common Issues

Quick Start

1. Install tfgrid-compose

2. Configure

3. Deploy & Use

Basic Commands

Command Shortcuts

Complete Workflow Example

How It Works

Documentation

Getting Started

Development

Patterns

Development

Running Tests

Troubleshooting

Contributing

License

Support

Links

About

Topics

Resources

License

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages