Skip to main content

Building a Multi-Tenant Restaurant Operating System from Scratch

One codebase. Four product surfaces. Real-time KDS, split billing, dynamic pricing, loyalty gamification, custom TLS automation, and a thermal print station — built solo.

19

Project Overview

RoleFull-Stack Developer
Reading Time~12 min read
Duration
Sep 25'Present
Team SizeSolo Developer
PlatformsWeb, Desktop (Electron)

The Problem: Restaurant businesses run on five or six disconnected tools — a separate POS, a separate menu app, a spreadsheet for inventory, WhatsApp for orders. Every handoff leaks money and data. The Solution: I designed and built Sofreh — a multi-tenant SaaS platform with a single unified data model spanning menus, real-time orders, kitchen display, inventory, loyalty, split billing, and reporting. The architecture handles public guest ordering, an authenticated establishment workspace, a platform admin console, and a standalone Electron print station, all from one Next.js App Router codebase. The Craft: The system includes non-trivial engineering: a middleware proxy that rewrites tenant subdomains and custom domains without a separate service, per-establishment SSE channels with automatic dead-listener cleanup, Redis-based distributed locks preventing race conditions in split billing and checkout, subscription entitlement snapshots with PostgreSQL advisory locking, a dynamic pricing engine with a 3-tier priority hierarchy, an ESC/POS thermal receipt generator, and a sales prediction model using 30-day momentum weighting.

The Challenge

Building a restaurant operating platform solo means solving a set of problems that individually are hard, but together are punishing. The real difficulty was making every system correct under concurrency, while keeping the codebase coherent enough to extend without regressions.

  • Multi-tenant isolation at every layer: Every query, every API response, every public menu URL must be scoped to the correct establishment. A data leak isn't a bug — it's a business-ending incident. Enforcement had to be structural, not a checklist.
  • Real-time without WebSocket infrastructure: Kitchen staff need order updates the instant a guest checks out. SSE is HTTP-native and proxy-friendly, but a naïve implementation leaks connections silently. Dead listeners accumulate and the channel map grows unbounded.
  • Race conditions in money flows: Two guests trying to pay simultaneously for the same split-bill share, two staff members placing overlapping orders — these need distributed coordination. A Redis distributed lock was non-negotiable.
  • Plan limits under concurrent write pressure: Subscription entitlements that cap menus, staff seats, or orders need to be enforced under concurrent writes. A naive read-check-write creates a TOCTOU window that lets tenants silently exceed their plan.
  • Custom domain TLS per tenant: Establishments can serve their menu from their own domain. That means DNS verification, Certbot certificate issuance, NGINX hot-reconfiguration, and teardown on plan downgrade — all automated, all safe to retry, all correct under failure.
  • Pricing that changes by the hour: Happy-hour discounts, category-wide promotions, and per-item overrides must resolve to a single effective price at checkout, with a clear priority hierarchy and support for rules that wrap past midnight.
  • A Persian RTL product, not a retrofitted LTR one: The entire product — guest menus, dashboards, modals, receipts — is in Persian and right-to-left. Logical alignment, bidirectional text isolation, and Persian financial formatting had to be structural from day one.

The Engineering

Every system below came from reading the actual problem, not from copying a pattern. The goal was correctness under failure, not elegance for its own sake.

The multi-tenant proxy — one rewrite rule, zero service duplication

The Next.js middleware proxy resolves tenant identity before any page or API code runs. A tenantSlugOf(hostname) function strips the subdomain from *.sofreh.net or *.localhost. For custom domains, it detects any valid hostname that isn't the platform root. It then rewrites the request internally to /public-menu/[slug-or-hostname] while injecting x-sofreh-tenant-kind and x-sofreh-tenant-host headers — no separate routing service, no per-tenant deployment.

  • Host header allowlist rejects unknown hosts at the edge — defense-in-depth behind NGINX's return 444
  • Legacy /public-menu/[slug] paths issue a 308 permanent redirect to the tenant subdomain — SEO equity preserved
  • Card URLs get a canonical host, private cache headers, and X-Robots-Tag: noindex
  • Suspicious requests (directory traversal, SQL injection, XSS patterns, double-encoded probes) blocked before auth runs
  • JWT verification is lazy-loaded — jose is only imported when a protected route is hit, keeping public routes fast
  • Per-request CSP nonce generated with crypto.getRandomValues, injected as x-nonce header for inline scripts

Per-establishment SSE pub/sub — real-time KDS with dead-listener cleanup

RealTimeEventService maintains a Map<establishmentId, Set<ClientListener>> in process memory. When a client connects, it registers and receives a CONNECTED handshake. When an order is placed, broadcast() iterates the listener set — failures are caught, dead listeners collected, and if the channel empties, the key is deleted from the map entirely.

  • Events: NEW_ORDER, ORDER_UPDATED, WAITER_CALLED, WAITER_CALL_UPDATED, SHUTTER_TOGGLED, PRINTER_ALERT
  • Dead listeners removed synchronously during broadcast — no background GC needed
  • Empty channels pruned from the map immediately — no unbounded memory growth
  • SSE is plain HTTP/1.1 — works through every reverse proxy with no upgrade negotiation
  • In-process broadcast is a direct function call — no Redis hop on the hot path

Redis distributed lock — checkout and split billing without race conditions

Both checkout and split billing use withDistributedLock(key, action) — a helper that acquires a Redis SET NX PX lock with a unique per-worker value, runs the action, then releases via a Lua script that atomically checks the value before deleting. If Redis is down, the lock fails closed: the operation is rejected rather than allowed through unguarded.

  • Checkout lock key: checkout:[establishmentId]:[customerId|tableNumber|ip]
  • Split billing lock key: split:initiate:[orderId]
  • Lua release: if GET key == lockValue then DEL key end — atomic, impossible to delete the wrong lock
  • Retries: 5 attempts with 100ms backoff before giving up
  • Lock TTL: 30 seconds — prevents eternal locks on crashed workers
  • Failure message is in Persian: users get a clear retry prompt, not an unhandled 500

Subscription entitlements — plan limits enforced under concurrent writes

Each establishment stores an entitlement snapshot — a JSON blob rebuilt from plan features on every plan change. When a write would push usage over a plan limit, requireLimitIncrement() takes a PostgreSQL advisory lock via pg_advisory_xact_lock(hashtext(key)) inside the transaction — making the check-and-increment atomic, even under concurrent writes from multiple requests.

  • Boolean features: can_use_customer_club, can_use_custom_domain, can_use_gamification…
  • Numeric limits: menu item cap, staff seat cap, order volume thresholds
  • Grace period: GRACE_PERIOD subscriptions retain full entitlements until the window closes
  • Entitlement snapshots are O(1) reads — one JSON parse per request, not a 4-table join
  • Feature check is a single call: subscriptionAccessControlService.requireFeature(id, 'can_use_gamification')

Dynamic pricing — happy-hour rules with a 3-tier priority hierarchy

DynamicPricingService.evaluateItemPrice() loads active rules for the establishment, filters to those active at the current time (including overnight windows like 23:00–02:00), then resolves a winner using a strict priority hierarchy: item-specific beats category-wide beats establishment-wide. The result is compared to the static item discount — the higher discount always wins.

  • Rules define daysOfWeek[], startTime, endTime, and discountPercentage
  • Overnight rules handled by inverting the time comparison when startTime > endTime
  • Priority: menuItemId match > categoryId match > no scope (establishment-wide)
  • Always takes max(dynamic discount, static discount) — promotions never make items more expensive
  • Price is evaluated server-side at checkout — no client-side manipulation possible

Recipe costing & menu engineering matrix

RecipeCostingService links each menu item to its ingredients with quantities and per-unit material costs, computing real-time COGS and margin percent. Items are then classified using the standard menu engineering matrix: STAR (high margin + high volume), PLOWHORSE, PUZZLE, and DOG — giving owners actionable pricing intelligence from their own sales data.

  • COGS: sum of (quantityUsed × material.costPerUnit) across all ingredients
  • Profitability matrix classifies every item as STAR / PLOWHORSE / PUZZLE / DOG
  • Order volume pulled from historical OrderItem records — not a static field, updates as sales occur
  • Distributed lock wraps COGS updates to prevent concurrent material-cost drift
  • Owners see per-item margin percent and can identify underpriced bestsellers in one view

Sales prediction — 30-day momentum model with day-of-week weighting

SalesPredictionService.predictTomorrowSales() fetches 30 days of completed orders, isolates those matching tomorrow's day of week, computes a day-of-week average, then applies a momentum factor from the trailing 7-day trend — clamped between 0.7× and 1.3× to prevent outlier distortion. An hourly distribution for the predicted order count is also generated. All output labels are in Persian.

  • Day-of-week average: isolates the same weekday across 30 historical days
  • Momentum: 7-day revenue average ÷ 30-day daily average, clamped to [0.7, 1.3]
  • Hourly distribution generated from historical order timestamps — shows when the rush will hit
  • Confidence score accompanies every prediction — owners know how reliable the estimate is
  • Safe zero-data path: returns a structured default when no order history exists

Gamification — probability-weighted spin wheel with transactional idempotency

GamificationService.spinWheel() checks the subscription entitlement, acquires a customer-level account lock, enforces daily spin limits, deducts the spin cost, then runs a weighted random draw with crypto.randomInt(total) over prize probability weights. Prize types (points, wallet credit, coupons, free items, VIP upgrades) each have their own write path. The entire sequence — cost deduction, prize award, history record — commits in one transaction. A requestKey deduplicates retries.

  • Prize types: POINTS, WALLET_BALANCE, COUPON, FREE_ITEM, VIP_UPGRADE
  • Weighted draw: crypto.randomInt(totalWeight) — no floating-point bias
  • requestKey deduplication: second call with the same key returns the same result without re-spinning
  • Daily spin cap enforced inside the transaction with a customer-level advisory lock
  • Coupon prizes cloned from a template with a 1-week validity if no expiry is set
  • Entire spin committed atomically — no partial state possible

ESC/POS thermal receipt generation — raw bytes, no print dialog

ThermalPrinterService.generateEscPosBuffer() constructs raw ESC/POS byte sequences for 80mm and 58mm thermal printers. It writes printer initialization (ESC @), center-align (ESC a 1), double-height header font (GS ! 0x11), left-aligned item lines, separator dashes, and totals — all as a Buffer sent directly to the printer port by the Electron app, with no browser print dialog and no user interaction.

  • Line width: 42 chars for 80mm, 32 chars for 58mm
  • Header: double-height establishment name via GS ! 0x11, reset with GS ! 0x00
  • Item lines: left-justified name, right-justified price, padded to line width
  • Totals: original price, discount amount, final price, payment method, tracking token
  • PRINTER_ALERT SSE event broadcast if the print station loses connection
  • Electron app calls this service directly — no browser, no dialog, no user action

Results & Impact

  • Services Built68 Services

    68 typed domain service classes — checkout, split billing, dynamic pricing, recipe costing, sales prediction, gamification, loyalty, referrals, custom domain TLS, push notifications, thermal printing — each isolated and wired through RegistryLocator for testability.

  • Schema Models38 Models

    The Prisma schema is composed from 38 focused domain files — orders, payments, split billing, inventory, loyalty, gamification, audit, TLS domains, push subscriptions — assembled into a single migration-safe schema. Every domain has a clear owner.

  • Tenant RoutingZero-Code

    The middleware proxy handles both subdomain tenants (*.sofreh.net) and custom-domain tenants through a single rewrite rule — no per-tenant deployment, no routing config per restaurant. A new establishment goes live the moment its slug is created.

  • Race ProtectionRedis Locks

    Every concurrent money flow — checkout, split billing, subscription limit increments — is protected by a Redis distributed lock (SET NX + Lua atomic release). Redis downtime fails closed: the operation is rejected, not silently allowed through.

  • Pricing Engine3-Tier Rules

    The dynamic pricing engine resolves a single effective price from item-specific, category-wide, and establishment-wide happy-hour rules — including overnight windows — and always applies the higher of the dynamic or static discount.

  • Print StationESC/POS Native

    The Electron desktop app generates raw ESC/POS byte buffers for 80mm and 58mm thermal printers and writes them directly to the printer port — no dialog, no user action. Orders are printed the moment the SSE event arrives.

Technical Architecture

Sofreh is a modular Next.js App Router application with 68 typed domain services, a 38-model composed Prisma schema, Redis for locking and real-time coordination, and a separate Electron print station — all from a single repository with CI architecture enforcement.

  • Middleware proxy (433 lines): Host validation, tenant rewrite, legacy redirect, card URL isolation, suspicious request blocking, lazy JWT verification, CSP nonce generation, security headers — before any page or API code runs.
  • 68 domain services: Checkout, SplitBill, DynamicPricing, RecipeCosting, SalesPrediction, Gamification, ThermalPrinter, EstablishmentDomain, SubscriptionEntitlement, RealTimeEvent… each isolated and wired through RegistryLocator for testability.
  • 38-model composable schema: order.prisma, split_payment.prisma, dynamic_pricing.prisma, gamification.prisma, customer_club.prisma… — 38 focused files composed into one migration-safe schema.
  • Redis roles: Session cache (best-effort before DB fallback), distributed lock store, rate limiter. Downtime degrades gracefully — locks fail closed, cache misses fall through to PostgreSQL.
  • SSE pub/sub: In-process Map<establishmentId, Set<listener>>. Broadcast is a direct function call — no Redis hop on the hot path. Dead listeners cleaned up synchronously.
  • Architecture CI gate: npm run architecture:check enforces that feature modules never import API internals, never call fetch ad hoc, and never use broad any. A governance ratchet enforces thin-edge ceilings.
  • Electron print station: Isolated Vite/Electron app, own lockfile. Subscribes to SSE identically to the browser KDS. Calls generateEscPosBuffer() and writes bytes to the printer port.

Architecture Decisions & Trade-offs

Decision principle

Every decision below came from reading the actual failure mode, not from a pattern checklist.

  1. Realtime transportADR / 01

    SSE over WebSockets for the KDS

    Restaurant networks sit behind NAT, cheap routers, and sometimes NGINX configs that don't handle WebSocket upgrades. SSE is plain HTTP/1.1 — it reconnects automatically, works through every proxy, and requires zero extra infrastructure. The KDS is read-only from the client; bidirectionality would have been pure complexity with no benefit.

  2. Scaling boundaryADR / 02

    In-process SSE map over Redis pub/sub

    With a single-worker deployment, in-process broadcast is a direct function call — no serialization, no network hop, no Redis dependency on the hot path. If the deployment ever scales horizontally, the SSE map becomes the migration target. Building for a problem that doesn't exist yet is technical debt, not engineering.

  3. Concurrency controlADR / 03

    PostgreSQL advisory locks for entitlement limits

    A naive read-check-write creates a TOCTOU window: two concurrent requests both read 'under limit', both pass, both write — silently exceeding the cap. pg_advisory_xact_lock(hashtext(key)) inside the transaction makes the check-and-increment atomic at the database level, with no separate locking service or serializable isolation level needed.

  4. Read-path designADR / 04

    Entitlement snapshots over live plan reads

    Reading the full plan feature list on every entitlement check means a join across subscription → plan → plan_features on every write. The entitlement state is snapshotted as a JSON blob and rebuilt only on plan changes. Reads become an O(1) JSON parse; plan changes are rare.

  5. Distributed lock safetyADR / 05

    Redis Lua script for lock release over a simple DEL

    A plain DEL has a race: if the TTL expires between acquire and release, a second worker acquires the lock before the first finishes — then the first's DEL deletes the second worker's lock. The Lua script atomically checks the value matches before deleting, making it structurally impossible to release the wrong lock.

Deployment & Infrastructure

Self-hosted, fully containerized, zero PaaS dependency for the data path. One Docker Compose file runs the entire production stack.

  • Docker Compose production stack: Next.js standalone container, PostgreSQL, Redis, NGINX, and GlitchTip (Sentry-compatible) as a single deployable unit — no separate infrastructure service.
  • NGINX edge: Terminates TLS, serves the wildcard platform certificate (*.sofreh.net), routes tenant custom domains, enforces security headers, and blocks unknown hosts with return 444 before Next.js sees them.
  • Custom domain TLS pipeline: Request domain → issue TXT verification token (_sofreh.[domain] = sofreh-verify-[token]) → DNS verified → Certbot issues certificate → atomic NGINX volume swap → teardown on plan downgrade via PENDING_DELETION status.
  • Prisma migrations applied from the deployed image: The running schema always matches deployed code — no manual migration step, no drift.
  • Redis: Session cache, rate limiter, distributed lock store. Never a source of truth — always a cache or coordination layer. Outage degrades gracefully.
  • GlitchTip / Sentry-compatible: Self-hosted error tracking, structured logging throughout the service layer, /api/health endpoint for container orchestration health checks.
  • CI governance gates: Architecture check, governance ratchet (boundary ratchets, thin-edge ceilings, suppression ledger), contract tests, typecheck, lint — broken code doesn't reach production.

What Building This Taught Me

Sofreh is the project where I learned that correctness is a harder engineering problem than capability. Adding a gamification spin wheel is an afternoon. Making sure the prize is never double-awarded, the cost never double-deducted, and the history always consistent — that's the actual work. The same pattern repeated across split billing, subscription limits, checkout, and custom domain issuance. Every system involving money or state transitions needs a locking strategy, an idempotency key, and a failure mode that fails closed, not open. The other deep lesson: multi-tenancy isn't a feature you add. It's a constraint that reshapes every query, every API boundary, every data model. When isolation is structural — enforced at the repository layer, not the route handler — regressions become structurally impossible, not just unlikely.

Key Takeaways

  • Fail closed, not open: Every distributed lock in Sofreh returns false when Redis is down — the operation is rejected with a clear message, not silently allowed through. A 503 is recoverable. Silent data corruption is not.
  • Multi-tenancy is a data model decision: Tenant isolation enforced in repositories means it's structurally impossible to write a route that leaks cross-tenant data. Enforcement at the route layer is a checklist. Enforcement at the query layer is architecture.
  • Idempotency keys are not optional: Spin wheel, split billing initiation, split payment — every operation a user might retry on a network error needs a key that makes the second call identical to the first.
  • Snapshots beat live reads for hot paths: One JSON parse per request instead of a 4-table join. Update the snapshot on plan change. Reads stay O(1) forever.
  • The proxy is the first line of product logic: Security headers, tenant resolution, auth redirect, legacy redirect, CSP nonce — all before any page code runs. A thin proxy that does these correctly is worth more than sophisticated page logic running on a bad request.

Share this case study