System Overview

What GPUForge Does

GPUForge solves the operational complexity of running shared GPU infrastructure for AI/ML workloads. Operators provision clusters, tenants consume GPU capacity, and the platform handles scheduling, quota enforcement, and billing — all from a real-time dashboard.

Three core services unified in one deploy:

• GPU-as-a-Service (GPUaaS) — Cluster/node/GPU registry, metrics ingestion, fleet health monitoring
• Inference as a Service — GPU allocation tracking, inference endpoint assignment, SLA monitoring
• Agentic AI Workflow Orchestration — Job submission, priority scheduling, tenant quota enforcement, multi-job pipelines

Target Users

Primary audience: GPU cloud operators — platform engineers, MLOps leads, and CTOs at companies running shared GPU infrastructure (cloud providers, research institutions, enterprise AI platforms).

Two dashboard planes serve distinct personas:

Target Workloads

Service 1 — GPU-as-a-Service (GPUaaS)

The foundation layer. Manages the physical GPU fleet lifecycle from node registration through real-time health monitoring.

Scope: cluster creation, node provisioning, GPU registration, metrics ingestion (utilization, memory, temperature, power), fleet health scoring, node status (online/offline/error), cluster-level capacity tracking.

Not owned: job scheduling logic, billing, tenant management.

Service 2 — Inference as a Service

GPU allocation and lifecycle for inference workloads. Tracks per-GPU assignment, inference endpoint status, and SLA metrics.

Scope: inference endpoint registry, GPU assignment to endpoints, endpoint health checks, per-endpoint resource usage, throughput metrics, latency tracking.

Not owned: model serving infrastructure (handled externally), job scheduling (delegated to GPUaaS scheduler).

Service 3 — Agentic AI Workflow Orchestration

The highest-level service. Coordinates multi-step AI workflows across GPU tenants — chain, loop, tool call patterns for autonomous agents.

Scope: job submission and lifecycle management, priority-based scheduling, tenant quota enforcement, multi-job pipeline tracking, workflow state machine (pending/running/completed/failed), retry logic, workflow logging.

Not owned: cluster/node management (GPUaaS), billing aggregation.

3a. Landing Page & SEO Pages

File: public/index.html (landing), public/use-cases/*.html, public/blog/*.html

The public-facing marketing surface. Landing page drives early-access signups and demo requests. SEO pages target specific use-case keywords (GPU cluster management, AI infrastructure monitoring, multi-tenant GPU platform). Client-side GA4 integration tracks page views, signup events, and scroll depth.

Key routes: GET /, GET /use-cases/*, GET /blog/*

Sub-components: hero with demo request modal, feature cards, contact sales section, request demo modal (POST /api/demo-requests → demo_requests table → Polsia email notification)

3b. Admin Dashboard

File: public/admin.html — Single-page app (SPA) served as static HTML, client-side JS fetches from /api/dashboard, /api/clusters, /api/alerts, /api/billing.

Fleet operator's command center. Key panels:

3c. Customer Dashboard

File: public/tenant.html — SPA served at /dashboard and /customer, auth-gated via localStorage session token. Tenants see only their own GPU allocations, jobs, and billing.

Key panels:

3d. REST API Layer (25+ Endpoints)

Location: routes/*.js — one Router per domain group. Entry point server.js mounts all at /api.

All endpoints are RESTful, JSON-only, and support API key authentication via Authorization: Bearer <key> header. Demo mode allows unauthenticated access for testing.

Endpoint groups:

See LLD for complete endpoint specs with request/response schemas.

3e. Job Scheduling Daemon

Location: jobs/scheduler-cycle.js + routes/scheduler.js

Runs every 30 seconds. Priority-based scheduler with tenant quota enforcement. Key behaviors:

• Selects idle GPUs (status = 'idle') across all active clusters
• Orders queued jobs by priority ascending (1 = highest)
• For each job: checks tenant quota (gpu_in_use + gpu_count ≤ gpu_quota)
• Assigns idle GPUs to eligible jobs with FOR UPDATE SKIP LOCKED (prevents double-assignment)
• Jobs that fail quota checks stay queued or flip to blocked if the reason is quota
• Emits usage_events for newly-completed jobs (GPU-hours consumed)
• Logs every cycle to scheduler_logs — audit trail of decisions

Deployment model: In-process setInterval guarded by POLSIA_IN_PROCESS_CRONS_ENABLED=true on Render. Blaxel shadow sets this to false; Blaxel triggers the same script via polsia.toml [[crons]].

// Guard pattern (server.js)
if (process.env.POLSIA_IN_PROCESS_CRONS_ENABLED === 'true') {
  setInterval(runSchedulerCycle, 30000);
  // Also runs on startup after 5s delay
}

3f. Fleet Alert System

Location: routes/alerts.js + db/alerts.js

Configurable threshold rules evaluated every 30s. Three built-in rule types:

Alert events stored in alert_events with JSONB details. Notification via Polsia email proxy when rule has notification_enabled = true.

3g. Demo Mode & Mock Auth

Location: routes/demo.js, db/auth.js

Demo mode provisions a complete GPU cluster environment for evaluation without operator setup. The operator dashboard (/admin) runs in demo mode by default when no live clusters exist.

Mock auth: session tokens stored in localStorage. Admin role = any login to /admin. Tenant role = login to /dashboard. API key auth uses SHA-256 hashed keys stored in api_keys table — only the 12-char prefix shown in UI; full key shown once at creation.

Login Flow → Role Routing → Dashboard

User navigates to /admin or /dashboard
         │
         ▼
   [Login Page]  /login
         │
         ├─ Admin login → session set (localStorage) → redirect /admin
         └─ Tenant login → session set (localStorage) → redirect /dashboard
         │
         ▼
   SPA loads, reads localStorage session token
         │
         ▼
   SPA calls GET /api/dashboard
         │
         ▼
   Express middleware (apiKeyAuth)
         │
         ├─ No Authorization header → Demo mode: serves all data (no auth gate)
         ├─ Valid Bearer key → lookupApiKey() → attach req.apiKey → next()
         └─ Invalid/revoked → 401 error response
         │
         ▼
   Route handler executes (e.g. GET /api/dashboard)
         │
         ▼
   db/ query functions (db/dashboard.js, db/clusters.js, etc.)
         │
         ▼
   pool.query() against Neon PostgreSQL
         │
         ▼
   JSON response returned to SPA
         │
         ▼
   SPA updates DOM (charts, tables, status cards)

Job Lifecycle Flow

1. SUBMIT
   POST /api/jobs { cluster_id, name, gpu_count, gpu_type, priority, tenant_id }
        │
        ▼
   Inline quota check (routes/jobs.js):
     SELECT gpu_quota FROM tenants WHERE id = tenant_id
     SELECT SUM(gpu_count) FROM jobs WHERE tenant_id = X AND status = 'running'
        │
        ├─ If current + requested > quota → job status = 'blocked', block_reason set
        ├─ If cluster idle_gpus < requested → job status = 'blocked', block_reason set
        └─ Otherwise → job status = 'queued'
        │
        ▼
   INSERT INTO jobs (...)

2. SCHEDULE (every 30s)
   Scheduler daemon runs runSchedulerCycle()
        │
        ▼
   SELECT idle GPUs: SELECT * FROM gpus WHERE status = 'idle' FOR UPDATE SKIP LOCKED
        │
        ▼
   SELECT queued jobs: SELECT * FROM jobs WHERE status = 'queued' ORDER BY priority ASC
        │
        ▼
   For each job:
     Check: tenant.gpu_in_use + job.gpu_count <= tenant.gpu_quota
        │
        ├─ YES → Assign N idle GPUs (set gpus.job_id, gpus.status = 'active')
                   Update job.status = 'running'
        └─ NO  → Leave as queued (or 'blocked' if quota was reason)
        │
        ▼
   INSERT INTO scheduler_logs (cycle_summary)
   INSERT INTO usage_events (for completed jobs: gpu_hours × rate)

3. COMPLETE
   PATCH /api/jobs/:id { status: 'completed' }
        │
        ▼
   Free allocated GPUs: UPDATE gpus SET job_id = NULL, status = 'idle'
   Emit usage_event: INSERT INTO usage_events (tenant, gpu_type, gpu_hours, cost)

Metrics Ingestion Flow

GPU Node agent runs on hardware, posts metrics every 60s:

   POST /api/clusters/metrics
   Authorization: Bearer gf_<cluster_api_key>
   Body: { metrics: [{ gpu_id, utilization_pct, memory_used_mb,
                       memory_total_mb, temperature_c, power_draw_w }, ...] }
        │
        ▼
   apiKeyAuth middleware validates cluster API key
        │
        ▼
   Bulk INSERT INTO gpu_metrics (recorded_at = NOW())
        │
        ▼
   For each GPU in payload:
     UPDATE gpus SET status =
       CASE WHEN utilization_pct > 10 THEN 'active' ELSE 'idle' END
       WHERE id = gpu_id
        │
        ▼
   Alert engine evaluates rules (every 30s):
     idle_gpu rule: COUNT(idle gpus) / total_gpus > threshold?
     health_degraded rule: temperature_c > 90 OR node unreachable?
        │
        ├─ Threshold crossed → INSERT INTO alert_events
        └─ notification_enabled → Polsia email proxy fires alert email

Billing Flow

1. USAGE CAPTURE (scheduler emits on job completion)
   INSERT INTO usage_events (tenant_id, gpu_type, gpu_hours, cost)
        │
        ▼
2. INVOICE GENERATION
   POST /api/tenants/:id/invoice  (or POST /api/billing/invoice/:tenantId)
        │
        ▼
   Aggregate usage_events for tenant (last billing period)
   Calculate: SUM(gpu_hours × rate) = total_amount
        │
        ▼
   Polsia Stripe proxy → create Stripe payment link
   INSERT INTO invoices (tenant_id, amount, status = 'pending', stripe_payment_url)
        │
        ▼
3. PAYMENT
   Tenant clicks Stripe payment link → pays on Stripe.com
   Polsia Stripe proxy → webhook / confirmation
   UPDATE invoices SET status = 'paid' WHERE id = invoice_id
        │
        ▼
4. RECONCILIATION
   GET /api/billing/summary → operator sees: revenue, outstanding, paid invoices
   GET /api/tenants/:id/invoices → tenant sees payment history

Core Tables

See LLD for full schema with all columns, types, constraints, and indexes.

Backend

Frontend

Infrastructure & Deployment

Project Structure

gpuforge/
├── server.js              # Entry point (172 lines — wiring only)
├── routes/
│   ├── clusters.js        # Cluster, node, GPU registry + metrics
│   ├── jobs.js            # Job submission + status update
│   ├── tenants.js         # Tenant CRUD + usage queries
│   ├── billing.js         # Invoice management + billing KPIs
│   ├── alerts.js          # Alert rules + event queries
│   ├── dashboard.js       # KPI aggregates, analytics
│   ├── api-keys.js        # Key generation + revocation
│   ├── scheduler.js       # Scheduler cycle trigger + logs
│   ├── analytics.js       # Page view stats
│   ├── demo.js            # Demo mode provisioning
│   ├── demo-requests.js   # Demo request form handler
│   └── misc.js            # GPU rates, early access signup
├── db/
│   ├── index.js           # new Pool() — only DB access point
│   ├── clusters.js        # Named query functions
│   ├── jobs.js
│   ├── tenants.js
│   ├── billing.js
│   ├── alerts.js
│   ├── dashboard.js
│   ├── api-keys.js        # SHA-256 key hashing + lookup
│   ├── analytics.js
│   ├── auth.js            # API key lookup + touchLastUsed
│   └── demo-requests.js
├── public/
│   ├── index.html         # Landing page
│   ├── admin.html         # Admin SPA
│   ├── tenant.html        # Customer SPA
│   ├── login.html         # Auth page
│   ├── pitch.html         # Pitch one-pager
│   ├── docs-hld.html      # This document
│   ├── docs-lld.html      # Low-Level Design
│   ├── docs-arch.html     # Architecture deep-dive
│   ├── docs.html          # API reference
│   ├── docs-hub.html      # Docs hub page
│   └── use-cases/         # SEO landing pages
├── migrations/             # node-pg-migrate SQL files
├── jobs/
│   └── scheduler-cycle.js  # Standalone scheduler runner
├── migrate.js             # Migration runner (npm run migrate)
├── polsia.toml            # Cron declarations for Blaxel
└── render.yaml            # Render service config

Performance Targets

Security Model

Current (v1): Mock auth with localStorage session tokens. API keys stored as SHA-256 hashes (bcrypt-ready). No production auth integration yet.

Production auth roadmap: SSO/SAML for operator accounts, JWT-based tenant auth, RBAC with per-tenant permission scoping, OAuth2 integration for cloud provider credentials.

Scalability Approach

Single-web-service architecture on Render. Scales horizontally via Render auto-scaling (CPU-based). Key design decisions for scale:

• Connection pooling: pg Pool (10 connections) shared across all route handlers. Neon handles branching and pooling at the DB layer.
• Scheduler distribution: The 30s scheduler runs in-process. For multi-instance deployment, FOR UPDATE SKIP LOCKED prevents duplicate GPU assignments. Blaxel will handle cron distribution.
• Metrics ingestion: Bulk INSERT (INSERT ... ON CONFLICT DO UPDATE) for GPU metrics — single round-trip per batch.
• Read scaling: Neon read replica endpoint available for read-heavy dashboards.
• Horizontal scale ceiling: Current architecture supports ~50 concurrent operators, ~200 tenants, ~10,000 GPUs per instance before needing a service split.

Availability & Reliability

Render Web Service

Neon Managed PostgreSQL

CI/CD via Polsia

Polsia's infra pipeline manages the full delivery flow:

• GitHub integration: Repo provisioned and connected to Render auto-deploy
• Render management: Service creation, env var management, log access via polsia_infra MCP
• Database provisioning: Neon database created and linked on first provision
• Shadow deployment: Blaxel shadow receives every deploy for pre-production validation
• Cron sync: polsia.toml [[crons]] entries synced to Blaxel as disabled triggers

Stripe Billing

Integration: Polsia Stripe proxy — no Stripe keys on GPUForge. All payment link creation goes through https://polsia.com/api/proxy/stripe/payment-link.

Flow: POST /api/tenants/:id/invoice → aggregates usage events → calls Polsia Stripe proxy → receives payment URL → stores in invoices.stripe_payment_url → tenant receives link, pays on Stripe → webhook confirms → invoice marked paid.

Future: Subscription billing (monthly plans), usage-based billing (auto-invoice monthly), webhook retry handling.

API Key Management

Storage: Full key hashed with SHA-256 before storage. Only 12-char prefix shown in UI. Full key shown once at creation, stored by user.

Endpoints: POST /api/api-keys (generate), GET /api/api-keys (list), DELETE /api/api-keys/:id (revoke), GET /api/api-keys/demo (get demo key for testing).

Auth middleware: Validates Authorization: Bearer <key> header against api_keys.key_hash. Revoked keys (revoked_at IS NOT NULL) return 401. Last used timestamp updated on every valid call.

Kubernetes / SLURM Integration (Future)

Integration point 1 — Kubernetes: GPU operators running k8s clusters can deploy a GPUForge node agent as a DaemonSet. The agent registers nodes and posts metrics via POST /api/clusters/nodes and POST /api/clusters/metrics. The scheduler daemon then orchestrates GPU allocation across the k8s cluster via the same job assignment API.

Integration point 2 — SLURM: On-premise HPC clusters running SLURM can integrate via a GPUForge SLURM plugin that translates SLURM job events into GPUForge job records. GPU allocation data flows back to SLURM via the node agent.

Integration point 3 — Ray / vLLM: Ray clusters and vLLM inference servers can register with GPUForge as special cluster types. GPUForge treats them as managed GPU pools with pre-assigned capacity.

Polsia Platform Integrations