Designing a surge pricing system like Uber using stream processing

Breaking down the system into functional requirements that can adapt in near real-time to fluctuating supply and demand

Shashank Mishra

Apr 17, 2025

1.Continuously track ride requests in various geographic zones.

2.Group users by geohash (e.g., 6-7 character length for ~1 km² resolution).

3.Maintain recent request counts using sliding windows (e.g., last 1–5 minutes).

4.Continuously track active driver availability in each zone.

5.Detect drivers becoming free or busy (status updates via stream).

6.Calculate demand/supply ratio per zone using a real-time streaming engine.

7.Trigger surge pricing when ratio exceeds a predefined threshold (e.g., 2:1).

8.Dynamically adjust geohash-based zones based on city density or activity.

9.Event-Driven Surge Updates Emit surge price updates to downstream services (pricing engine, user apps).

10.Surge pricing decays smoothly when demand drops (no sharp falls)

NFR

1.Surge multiplier updates should be calculated and reflected to users within < 1-2 seconds.

2.The system must be highly available (99.99% uptime) since pricing impacts both user experience and revenue.

3.Must scale horizontally to support millions of users and drivers concurrently.

4.Must recover gracefully from node failures in stream processing (Flink/Spark with checkpointing), basically system is fault tolerant

5.Surge calculations should be deterministic and consistent across services (pricing engine, user app, etc.).

Estimates

Uber-scale system in a large country like India or US

Peak: 5 million daily rides

Assume 4x that in ride requests (due to retries, unfulfilled ones): 20 million ride requests/day

Driver availability status updates: 10 million/day

Total events per day: ~30 million
5k events/se

API’s

1.POST /events/ride-requested

Triggered when a user requests a ride. Used by stream processor to increase demand count in the zone.

request {
  "user_id": "user_123",
  "timestamp": "2025-04-15T10:20:00Z",
  "location": {
    "lat": 37.7749,
    "lng": -122.4194
  }
}
response :202 accepted

2.POST /events/driver-status-update

Fired when driver becomes available/unavailable. Updates supply count.

{
  "driver_id": "driver_456",
  "status": "available",  // or "unavailable"
  "timestamp": "2025-04-15T10:20:05Z",
  "location": {
    "lat": 37.7750,
    "lng": -122.4185
  }
}

3.GET /surge/zones/{zone_id}
Used by pricing engine to fetch surge multiplier for a geohash zone.

📤 Request:
GET /surge/zones/9q8yyz (where 9q8yyz is geohash)

📥 Response:

{
  "zone_id": "9q8yyz",
  "surge_multiplier": 2.1,
  "updated_at": "2025-04-15T10:20:07Z"
}

4.GET /pricing/estimate

GET /pricing/estimate?pickup_lat=37.7749&pickup_lng=-122.4194&drop_lat=37.7849&drop_lng=-122.4094
{
  "base_fare": 120,
  "surge_multiplier": 1.8,
  "total_estimate": 216,
  "currency": "USD",
  "zone_id": "9q8yyz",
  "surge_applied_at": "2025-04-15T10:20:09Z"
}

5.GET /driver/surge-hotspots

Let driver see nearby zones with high surge

GET /driver/surge-hotspots?driver_lat=37.7749&driver_lng=-122.4194

[
  {
    "zone_id": "9q8yyz",
    "surge_multiplier": 2.3,
    "center": { "lat": 37.7750, "lng": -122.4190 }
  },
  {
    "zone_id": "9q8yyx",
    "surge_multiplier": 1.7,
    "center": { "lat": 37.7740, "lng": -122.4200 }
  }
]

6.POST /surge/recalculate

request {
  "zone_id": "9q8yyz"
}
response {
  "status": "recalculation_triggered",
  "timestamp": "2025-04-15T10:21:00Z"
}

7.POST /admin/surge-config

{
  "zone_id": "9q8yyz",
  "demand_supply_threshold": 2.5,
  "max_surge": 3.0,
  "smoothing_window_minutes": 5
}
{
  "message": "Configuration updated successfully.",
  "zone_id": "9q8yyz"
}

8.GET /admin/surge-history?zone_id=9q8yyz

[
  { "timestamp": "2025-04-15T10:00:00Z", "surge": 1.0 },
  { "timestamp": "2025-04-15T10:05:00Z", "surge": 1.5 },
  { "timestamp": "2025-04-15T10:10:00Z", "surge": 2.0 },
  { "timestamp": "2025-04-15T10:15:00Z", "surge": 2.3 }
]

Database Schema

let's design the databases and schema needed for the Surge Pricing System, keeping in mind stream processing, fast reads for pricing, and persistent storage for analytics and audit.

We'll divide into:

In-memory DB for fast access (e.g., Redis)
Durable DB for audit/history (e.g., Cassandra or PostgreSQL)
Config store for admin thresholds (e.g., MySQL/PostgreSQL)

1. Redis (or RocksDB State Store in Flink) — For Real-time Surge State

Use Redis as a key-value store for:

Current surge multiplier
Demand/supply counts
Last updated timestamp

🔧 Schema Design (Key → Value):

Key: surge:zone:{zone_id}

{
  "zone_id": "9q8yyz",
  "ride_requests": 23,
  "available_drivers": 5,
  "surge_multiplier": 2.3,
  "last_updated": "2025-04-15T10:25:00Z"
}

PostgreSQL / MySQL Schema (Config & Metadata)

-- Surge Configurations per Zone
CREATE TABLE surge_config (
    zone_id VARCHAR(20) PRIMARY KEY,
    demand_supply_threshold DOUBLE PRECISION NOT NULL,
    max_surge DOUBLE PRECISION NOT NULL DEFAULT 3.0,
    smoothing_window_min INTEGER NOT NULL DEFAULT 5,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

-- Zone Metadata (for mapping geohash to human-readable info)
CREATE TABLE zone_metadata (
    zone_id VARCHAR(20) PRIMARY KEY,
    city_name VARCHAR(100),
    center_lat DOUBLE PRECISION,
    center_lng DOUBLE PRECISION
);

Cassandra Schema (For Surge History)

-- Surge multiplier time-series per zone
CREATE TABLE surge_history (
    zone_id TEXT,
    timestamp TIMESTAMP,
    surge_multiplier DOUBLE,
    ride_requests INT,
    available_drivers INT,
    PRIMARY KEY (zone_id, timestamp)
) WITH CLUSTERING ORDER BY (timestamp DESC);

Redis Key Format (for Fast Real-time Access)

Key: surge:zone:{zone_id}
Value (stored as a JSON or hash):

{
  "zone_id": "9q8yyz",
  "ride_requests": 23,
  "available_drivers": 5,
  "surge_multiplier": 2.3,
  "last_updated": "2025-04-15T10:25:00Z"
}

HLD

          +-----------------+           +------------------+
          |   Rider App     |           |   Driver App     |
          +--------+--------+           +--------+---------+
                   |                             |
     POST /ride-request              POST /driver-status-update
                   |                             |
          +--------v--------+           +--------v--------+
          | Rider Service   |           | Driver Service  |
          +--------+--------+           +--------+--------+
                   |                             |
              Kafka Pub                     Kafka Pub
                   |                             |
                   +-------------+---------------+
                                 |
                       +---------v----------+
                       |   Surge Engine     |  <-- (Flink / Spark Streaming)
                       +---------+----------+
                                 |
            +----------------+----------------+
            |                                 |
     +------v-----+                  +--------v--------+
     |   Redis    |  <--- Fast R/W   |   Cassandra     | <-- Historical data
     +------------+                  +-----------------+
             |
  +----------v-----------+
  |   Pricing Service     | <-- Rider App calls for fare estimate
  +-----------------------+

Shashank’s Substack

Discussion about this post