Microservices platform built with NestJS, featuring event-driven architecture, gRPC communication, and GraphQL API
- Jobber - Microservices Platform
Jobber is a modern, scalable microservices platform built with NestJS and Nx monorepo. It demonstrates enterprise-grade architecture patterns including:
- Event-Driven Architecture using Apache Pulsar for asynchronous communication
- gRPC for high-performance inter-service communication
- GraphQL API for flexible client interactions
- Multi-database strategy with Prisma and Drizzle ORM
- Background job processing with a distributed task execution system
- Containerized deployment with Docker and Kubernetes (Helm)
The Jobber platform follows a microservices architecture with three primary communication patterns: GraphQL for client APIs, gRPC for inter-service communication, and Apache Pulsar for asynchronous job processing.
┌─────────────────────────────────────────────────────────────────────────┐
│ CLIENT APPLICATIONS │
│ (Web Browser, Mobile App, Postman) │
└────────────────────────────┬────────────────────────────────────────────┘
│
│ (GraphQL)
│ Mutations & Queries
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ GRAPHQL API LAYER │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Auth Service │ │ Jobs Service │ │ Products Service │ │
│ │ (Port 3000) │ │ (Port 3001) │ │ (Port 3002) │ │
│ ├──────────────────┤ ├──────────────────┤ ├──────────────────┤ │
│ │ • register() │ │ • createJob() │ │ • getProducts() │ │
│ │ • login() │ │ • uploadFile() │ │ • addProduct() │ │
│ │ • getUser() │ │ • getJobs() │ │ • updateStock() │ │
│ └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ │
└───────────┼──────────────────────┼──────────────────────┼───────────────┘
│ │ │
│ │ │
│ ┌────────────────────┘ │
│ │ gRPC: Authenticate(token) │
│ │ Returns: User{id, email} │
▼ ▼ ▼
┌────────────────────────────────────────────────────────────────────────┐
│ BUSINESS LOGIC LAYER │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Auth Microservice│ │ Jobs Microservice│ │Products Micro- │ │
│ │ (gRPC: 50051) │ │ (gRPC: 50052) │ │service (50053) │ │
│ ├──────────────────┤ ├──────────────────┤ ├──────────────────┤ │
│ │ • Hash passwords │ │ • Create tasks │ │ • Manage catalog │ │
│ │ • Verify tokens │ │ • Store uploads │ │ • Track stock │ │
│ │ • Manage users │ │ • Schedule jobs │ │ • Price updates │ │
│ │ │ │ │ │ │ │
│ │ Publishes: │ │ Publishes: │ │ Subscribes: │ │
│ │ - UserEvents │ │ - JobCreated ────┼───┼──► ProductLoad │ │
│ └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ │
└───────────┼──────────────────────┼──────────────────────┼──────────────┘
│ │ │
│ │ Pulsar Topic │
│ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ Executor Service │ │
│ │ │ │
│ │ • Consumes jobs │ │
│ │ • Executes: │ │
│ │ - FibonacciJob │ │
│ │ - LoadProductsJob │ │
│ │ - ... │ │
│ │ │ │
│ │ • Reports results │ │
│ └──────────┬──────────┘ │
│ │ │
│ │ Publish results │
│ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ Apache Pulsar │ │
│ │ Message Broker │ │
│ │ (Port 6650) │ │
│ │ │ │
│ │ Topics: │ │
│ │ • job-events │ │
│ │ • job-results │ │
│ │ • user-events │ │
│ └─────────────────────┘ │
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ PostgreSQL │ │ PostgreSQL │
│ Auth Database │ │ Other Databases │
└─────────────────────────┘ └─────────────────────────┘
✅ Separation of Concerns: Each service has a single responsibility
✅ Event-Driven: Asynchronous job processing decouples services
✅ API Gateway Pattern: GraphQL endpoints act as unified client interface
✅ Service Discovery: gRPC enables typed inter-service communication
✅ Database Per Service: Each microservice owns its data
✅ Message Queue: Pulsar ensures reliable, ordered job processing
✅ Scalability: Executor services can be horizontally scaled
Responsibilities:
- User authentication and authorization
- JWT token management
- User registration and login
- Session management
Technology:
- Database: PostgreSQL with Prisma ORM
- API: GraphQL (Apollo Server)
- Communication: gRPC server for internal auth verification
- Port: HTTP/GraphQL endpoint + gRPC endpoint
Responsibilities:
- Job/task definition and management
- File upload handling
- Task scheduling and coordination
- Integration with executor service
Technology:
- Database: PostgreSQL with Prisma ORM
- API: GraphQL
- Communication: gRPC client (Auth), gRPC server, Pulsar producer
- Features: File upload system, job orchestration
Available Job Types:
The system currently implements 2 demonstration jobs to showcase the architecture:
- Fibonacci Job - Computes Fibonacci sequences asynchronously
- Load Products Job - Imports product data from external sources
Extensibility: The architecture is designed to be highly scalable and easily adaptable for new job types. Simply create a new job class implementing the job interface, and the system will automatically discover and execute it through the executor service.
Responsibilities:
- Product catalog management
- Inventory tracking
- Product CRUD operations
Technology:
- Database: PostgreSQL with Drizzle ORM
- API: GraphQL
- Communication: gRPC server
- Data: Seed data from
data/products.json
Responsibilities:
- Background job execution
- Distributed task processing
- Job result reporting
- Processes jobs published to Apache Pulsar
Technology:
- Communication: Pulsar consumer
- Features: Worker pool, job execution engine
Job Processing Flow:
- Jobs service publishes job events to Pulsar
- Executor consumes events from job queue
- Jobs are executed in isolated worker processes
- Results are published back through Pulsar
- Jobs service updates status in the database
Current Job Implementations:
- Fibonacci Calculator - Demonstrates compute-intensive operations
- Product Loader - Demonstrates data import operations
💡 Note: The system is architected to seamlessly support additional job types. New jobs can be added without modifying the executor core - simply implement the job interface and the system will handle discovery and execution automatically.
- Node.js (v18 or higher)
- npm or yarn
- Docker & Docker Compose
- Protocol Buffers Compiler (protoc)
- Clone the repository
git clone https://github.com/MarwanRadwan7/node-microservices.git
cd jobber- Install dependencies
npm install- Start infrastructure services
docker-compose up -dThis starts:
- PostgreSQL (port 5432)
- Apache Pulsar (port 6650)
- Set up environment variables
Create .env files for each service:
# apps/auth/.env
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/auth"
AUTH_GRPC_SERVICE_URL="0.0.0.0:50051"
# apps/jobs/.env
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/jobs"
JOBS_GRPC_SERVICE_URL="0.0.0.0:50052"
AUTH_GRPC_SERVICE_URL="localhost:50051"
# apps/products/.env
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/products"
PRODUCTS_GRPC_SERVICE_URL="0.0.0.0:50053"- Run database migrations
# Auth service
npx nx run auth:prisma-migrate
# Jobs service
npx nx run jobs:prisma-migrate
# Products service (Drizzle)
npx nx run products:drizzle-push- Generate gRPC types
npx nx run grpc:generate-protoStart all services in development mode:
npx nx run-many --target=serve --all --parallelStart individual services:
# Auth service
npx nx serve auth
# Jobs service
npx nx serve jobs
# Products service
npx nx serve products
# Executor service
npx nx serve executorBuild all services:
npx nx run-many --target=build --allBuild individual service:
npx nx build authRun all tests:
npx nx run-many --target=test --allRun tests for a specific service:
npx nx test authBuild Docker images:
# Build all services
docker-compose build
# Build specific service
docker build -f apps/auth/Dockerfile -t jobber-auth .Run with Docker Compose:
docker-compose upUsing Helm:
# Install the Helm chart
helm install jobber ./charts/jobber
# Upgrade existing deployment
helm upgrade jobber ./charts/jobber
# Uninstall
helm uninstall jobberChart Configuration:
The Helm chart (charts/jobber/) includes:
- Apache Pulsar cluster configuration
- PostgreSQL database
- Service deployments for all microservices
- Ingress configuration
- ConfigMaps and Secrets
Edit charts/jobber/values.yaml to customize:
- Replica counts
- Resource limits
- Image tags
- Environment variables
Built with ❤️ using modern technologies and best practices