⚡ 1M RPS Architecture Lab

A production-grade, self-contained 1 Million Requests-Per-Second (1M RPS) architecture laboratory. This project lets you build, horizontally scale, load test, and observe a high-concurrency microservices architecture entirely inside Docker without modifying your host Linux environment.

🏛️ Architecture Overview

                           ┌───────────────────────────┐
                           │   k6 Distributed Load     │
                           │   Tester (Dockerized)     │
                           └─────────────┬─────────────┘
                                         │ HTTP Ingress (Keep-Alive Pools)
                                         ▼
                           ┌───────────────────────────┐
                           │     HAProxy 3.x (LB)      │
                           │  - Dynamic DNS Discovery  │
                           │  - Built-in Prometheus    │
                           │  - Health-Checked Routing │
                           └─────────────┬─────────────┘
                                         │ Round-Robin Load Distribution
                   ┌─────────────────────┼─────────────────────┐
                   ▼                     ▼                     ▼
           ┌───────────────┐     ┌───────────────┐     ┌───────────────┐
           │  Fastify API  │     │  Fastify API  │     │  Fastify API  │
           │  (Node.js 22) │     │  (Node.js 22) │     │  (Node.js 22) │
           │   Worker #1   │     │   Worker #2   │     │   Worker #N   │
           └───────┬───────┘     └───────┬───────┘     └───────┬───────┘
                   │                     │                     │
                   └─────────────────────┼─────────────────────┘
                                         │
                    ┌────────────────────┴────────────────────┐
                    │                                         │
                    ▼                                         ▼
           ┌─────────────────┐                       ┌─────────────────┐
           │  Redis 7 Cache  │                       │   Prometheus    │
           │  (In-Memory     │                       │  (Scrapes API   │
           │   Fast-Path)    │                       │  & HAProxy)     │
           └─────────────────┘                       └────────┬────────┘
                                                              │
                                                              ▼
                                                     ┌─────────────────┐
                                                     │  Grafana Live   │
                                                     │  Dashboard      │
                                                     │ (Auto-Loaded)   │
                                                     └─────────────────┘

Core Architecture Components

Fastify API Cluster (api/): High-performance Node.js v22 runtime with zero unnecessary logging overhead in benchmark mode, efficient JSON serialization, custom Prometheus instrumentation (prom-client), and connection keep-alive pooling.
HAProxy Layer 7 Load Balancer (haproxy/): Configured with maxconn 1,000,000, dynamic Docker DNS discovery (127.0.0.11) supporting auto-scaling up to 20 replicas without restart, and native Prometheus metrics exposition.
Redis Caching Layer (redis/): In-memory cache demonstrating sub-millisecond fast-path retrieval and database offloading.
Prometheus Monitoring (prometheus/): Telemetry server scraping HAProxy frontend/backend stats and Fastify per-worker metrics at 2-second intervals.
Grafana Real-Time Dashboard (grafana/): Pre-provisioned, zero-configuration live telemetry visualizing throughput (RPS), P50/P95/P99 latency percentiles, error rates, per-node traffic distribution, and memory/CPU utilization.
k6 Load Generator (k6/): Arrival-rate and staged ramping load test suites built to stress-test the cluster and identify the exact saturation ceiling of the underlying hardware.

⚡ Quick Start

1. Launch the Lab

Start all services with 3 API replicas with a single command:

docker compose up -d --build --scale api=3

(or use make up)

2. Access the Endpoints

Service	URL	Purpose
API Traffic Ingress	http://localhost:8080/api/test	High-throughput endpoint load balanced by HAProxy
Grafana Telemetry	http://localhost:3000	Live real-time telemetry dashboard (No login required)
HAProxy Live Stats	http://localhost:8404/stats	HAProxy backend server status and session table
Prometheus Web UI	http://localhost:9090	PromQL query engine and target scraper status

🧪 Step-by-Step Demonstration Guide

Step 1: Health Check & Verification

Verify the HAProxy load balancer and API nodes are healthy:

curl http://localhost:8080/health

Expected response:

{ "status": "ok", "pid": 1, "uptime": 12.4, "redis": "connected" }

Test round-robin distribution across workers:

for i in {1..5}; do curl -s http://localhost:8080/api/test; echo; done

Step 2: Run a Smoke Test

Execute a 10-second sanity check at 100 RPS:

make smoke

(or docker compose --profile loadtest run --rm k6 run /scripts/smoke.js)

Step 3: Run a High-Throughput Steady Load Test

Run a steady-state constant arrival-rate test (e.g., 10,000 RPS for 30 seconds):

make load TARGET_RPS=10000 DURATION=30s

Open http://localhost:3000 in your browser to watch the real-time throughput and latency graphs during the run.

Step 4: Staged Ramping Test (Hardware Saturation Curve)

To discover your hardware's exact saturation ceiling, execute the staged ramp-up test:

make ramp

The test ramps progressively: 1,000 RPS ➔ 5,000 RPS ➔ 10,000 RPS ➔ 25,000 RPS ➔ 50,000 RPS ➔ 100,000 RPS ➔ 250,000 RPS ➔ 500,000 RPS ➔ 1,000,000 RPS

Note on Target vs. Achieved RPS: Requesting 1,000,000 RPS with k6 tells the load tester to attempt that arrival rate. On a standard laptop/workstation, CPU and context switching will saturate earlier (e.g., 30k–70k RPS). The key learning is measuring the exact point where P95 latency and error rate deviate from linear behavior.

Step 5: Redis Cache Hit vs. Miss Benchmark

Benchmark how an in-memory caching layer protects backend systems under heavy load:

make cache-test

This runs against /api/product/:id with a 90% cache-hit distribution, demonstrating sub-millisecond response times for cached records.

Step 6: Automated Horizontal Scaling Experiment

To prove horizontal scaling efficiency, run the automated benchmark runner:

./benchmark.sh

(or make bench)

This script automatically scales the API from 1 to 4 replicas, executes benchmark suites, and produces a Markdown performance table:

=================================================================
        1M RPS Lab: Horizontal Scaling Benchmark Runner         
=================================================================
API Replicas    Target RPS      Achieved RPS    P95 Latency     P99 Latency     Errors      
--------------------------------------------------------------------------------------
1               15000 req/s     11420 req/s     8.42ms          14.10ms         0.00%       
2               15000 req/s     14980 req/s     2.15ms          4.80ms          0.00%       
3               15000 req/s     15000 req/s     1.32ms          2.40ms          0.00%       
4               15000 req/s     15000 req/s     0.94ms          1.80ms          0.00%       
--------------------------------------------------------------------------------------

Results are saved automatically to benchmark_results.md.

📊 Live Observability Dashboard

Grafana is pre-configured to automatically load the 1M RPS Architecture Telemetry dashboard on startup:

Ingress Rate (HAProxy): Total incoming HTTP requests per second (sum(rate(haproxy_frontend_http_requests_total[10s]))).
API Processed Throughput: Total completed requests across all API workers (sum(rate(http_requests_total[10s]))).
Latency Percentiles: P50 (median), P95, and P99 latency in milliseconds computed from Prometheus histograms.
Round-Robin Distribution: Per-node breakdown proving uniform traffic distribution across worker containers.
HTTP Status Codes: Real-time stacked chart of 2xx, 4xx, and 5xx responses.
Worker Runtime Telemetry: Resident Memory (RSS), Process CPU utilization, and Event Loop Lag.

📂 Project Directory Structure

1m-req/
├── docker-compose.yml           # Complete containerized service definition
├── Makefile                     # Convenient CLI commands
├── benchmark.sh                 # Automated scaling matrix benchmark runner
├── README.md                    # Project documentation and guide
├── plan.md                      # Original architecture design document
├── api/
│   ├── Dockerfile               # Node.js 22 alpine optimized image
│   ├── package.json             # Fastify, prom-client, and ioredis
│   └── server.js                # High-throughput API implementation
├── haproxy/
│   └── haproxy.cfg              # HAProxy config with DNS discovery & Prometheus exporter
├── k6/
│   └── scripts/
│       ├── smoke.js             # 100 RPS sanity test
│       ├── constant.js          # Configurable constant-rate load test
│       ├── ramp.js              # Multi-stage 1M RPS ramp-up stress test
│       └── cache_benchmark.js   # Redis cache hit vs miss benchmark
├── prometheus/
│   └── prometheus.yml           # Scrape targets (HAProxy, API pods, Prometheus)
└── grafana/
    ├── dashboards/
    │   └── 1m_rps_dashboard.json # Pre-built rich Grafana dashboard
    └── provisioning/
        ├── dashboards/dashboards.yaml   # Auto-provisioning dashboard provider
        └── datasources/prometheus.yaml  # Auto-provisioning Prometheus datasource

🛠️ Handy Make Commands

Command	Action
`make up`	Build and start all services with 3 API replicas
`make down`	Stop all services
`make smoke`	Run 10-second smoke test
`make load TARGET_RPS=10000`	Run steady-state benchmark
`make ramp`	Run staged stress test up to 1M RPS target
`make cache-test`	Run Redis cache benchmark
`make scale REPLICAS=5`	Scale API cluster to 5 containers dynamically
`make bench`	Run automated horizontal scaling benchmark
`make logs`	Stream logs from all containers
`make clean`	Stop containers and remove volumes

🧠 System Design Takeaways for High-Throughput Architectures

Target RPS ≠ Achieved RPS: Load generation tools can request arbitrarily high arrival rates, but true capacity is defined by the server's processed requests and sub-10ms P95 latency bounds.
Dynamic Service Discovery: HAProxy's DNS resolution allows application nodes to scale out dynamically without reloading or dropping active TCP connections.
Keep-Alive & Connection Pooling: Reusing established TCP connections between load balancers and upstream workers prevents TCP ephemeral port exhaustion.
Caching Fast-Paths: Adding an in-memory cache layer (Redis) offloads ~90%+ of repetitive queries, preventing downstream bottlenecks.
Observability Overhead: Keeping metrics collection lightweight (histogram bucket optimization, disabling verbose HTTP request logging) ensures telemetry does not degrade application throughput.

Navigation Menu

Repository navigation

About

Repository files navigation

⚡ 1M RPS Architecture Lab

🏛️ Architecture Overview

Core Architecture Components

⚡ Quick Start

1. Launch the Lab

2. Access the Endpoints

🧪 Step-by-Step Demonstration Guide

Step 1: Health Check & Verification

Step 2: Run a Smoke Test

Step 3: Run a High-Throughput Steady Load Test

Step 4: Staged Ramping Test (Hardware Saturation Curve)

Step 5: Redis Cache Hit vs. Miss Benchmark

Step 6: Automated Horizontal Scaling Experiment

📊 Live Observability Dashboard

📂 Project Directory Structure

🛠️ Handy Make Commands

🧠 System Design Takeaways for High-Throughput Architectures

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
api		api
grafana		grafana
haproxy		haproxy
k6/scripts		k6/scripts
prometheus		prometheus
.gitignore		.gitignore
Makefile		Makefile
README.md		README.md
benchmark.sh		benchmark.sh
benchmark_results.md		benchmark_results.md
docker-compose.yml		docker-compose.yml

Latest commit

History

Folders and files

Repository files navigation

⚡ 1M RPS Architecture Lab

🏛️ Architecture Overview

Core Architecture Components

⚡ Quick Start

1. Launch the Lab

2. Access the Endpoints

🧪 Step-by-Step Demonstration Guide

Step 1: Health Check & Verification

Step 2: Run a Smoke Test

Step 3: Run a High-Throughput Steady Load Test

Step 4: Staged Ramping Test (Hardware Saturation Curve)

Step 5: Redis Cache Hit vs. Miss Benchmark

Step 6: Automated Horizontal Scaling Experiment

📊 Live Observability Dashboard

📂 Project Directory Structure

🛠️ Handy Make Commands

🧠 System Design Takeaways for High-Throughput Architectures

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages