Design google street view in real time

Designing Google Street View in real-time is a complex and ambitious system that involves capturing, processing, and serving real-time panoramic imagery of streets to users across the globe.

May 22, 2025

1.Continuously capture panoramic images and videos from streets using cameras mounted on vehicles, drones, or stationary infrastructure.

2.Support multiple image resolutions (e.g., low, medium, high) based on bandwidth and user needs.

3.Process the captured images/videos in near real-time to:

Stitch multiple camera feeds into a 360° panoramic view.
Blur faces and license plates for privacy.
Compress images for fast delivery.

4.Ingest data from thousands of mobile devices (e.g., vehicles) simultaneously.

5.Match real-time imagery to specific GPS coordinates and roads.

6.Support spatial queries like:

“Show me the live view at [latitude, longitude]”
“Move forward on this road”

NFR

1.System should handle Millions of concurrent users globally.

2.Thousands of simultaneous real-time video/image streams from street-level devices (vehicles, drones, etc.).

3.High availability: 99.99% uptime for user-facing services.

4.Low latency streaming: Image data should be available to users within a few seconds of capture.

5.Retry failed uploads and handle intermittent network connectivity from moving vehicles.(reliability)

6.Retain raw and processed imagery reliably in cloud storage (e.g., durability of 11 9’s like in S3).(durability).

API’s

1.POST /v1/ingest/image

Upload a single image frame from a vehicle camera.

Request Body:
{
  "vehicle_id": "string",
  "timestamp": "ISO8601",
  "latitude": 37.7749,
  "longitude": -122.4194,
  "heading": 90,
  "image_data": "base64-encoded or multipart image",
  "camera_id": "front_left"
}
Response:
{
  "status": "success",
  "image_id": "uuid"
}

2.POST /v1/ingest/stream/initiate

Initiate a live video stream session.

Request Body:
{
  "vehicle_id": "string",
  "latitude": 37.7749,
  "longitude": -122.4194
}
Response:
{
  "stream_url": "rtmp://streaming-node/abcd1234",
  "session_id": "uuid"
}

3.POST /v1/process/image

Internal API for image processing (e.g., stitching, blurring).

Request Body:
{
  "image_id": "uuid",
  "tasks": ["stitch", "blur_faces", "geo_tag"]
}
Response:
{
  "status": "processing_started"
}

4.GET /v1/view/live

Fetch a real-time panoramic image for a specific location.

GET /v1/view/live?lat=37.7749&lng=-122.4194

{
  "image_url": "https://cdn.service.com/live/12345.jpg",
  "timestamp": "2025-05-21T12:05:34Z",
  "location": {
    "latitude": 37.7749,
    "longitude": -122.4194
  }
}

`5.GET /v1/view/navigation`

Get adjacent views (forward, backward, left, right).

GET /v1/view/navigation?image_id=12345

{
  "forward": { "image_id": "12346", "url": "..." },
  "backward": { "image_id": "12344", "url": "..." },
  "left": { "image_id": "12347", "url": "..." },
  "right": { "image_id": "12348", "url": "..." }
}

6.GET /v1/view/history

Get recent image history for a given point.

GET /v1/view/history?lat=37.7749&lng=-122.4194&limit=5

[
  { "timestamp": "...", "image_url": "..." },
  ...
]

7.GET /v1/coverage

Get areas where real-time Street View is currently active.

GET /v1/coverage?bbox=west,south,east,north

{
  "active_areas": [
    { "lat": 37.77, "lng": -122.41, "updated_at": "..." },
    ...
  ]
}

8.POST /v1/report/image

Report a privacy or content issue.

Request Body:
{
  "image_id": "12345",
  "issue_type": "privacy_violation",
  "description": "Blur is missing for a person’s face."
}

`9.POST /v1/moderate/image`

Admin-only action to manually blur/remove content.

Request Body:
{
  "image_id": "12345",
  "action": "remove"
}

10.GET /v1/analytics/traffic

Monitor user demand and viewer counts for specific regions.

GET /v1/analytics/traffic?region=san_francisco

{
  "vehicle_id": "string",
  "timestamp": "ISO8601",
  "speed": 32,
  "heading": 90,
  "gps": { "lat": ..., "lng": ... }
}

Databases

Relational DB: For structured metadata (PostgreSQL, MySQL).
Time-series DB: For telemetry data (e.g., TimescaleDB, InfluxDB).
Object Storage: For large image/video blobs (e.g., AWS S3, GCP Cloud Storage).
Search + Geo Indexing: For fast geo-queries (e.g., Elasticsearch, PostGIS

CREATE TABLE images (
    image_id UUID PRIMARY KEY,
    vehicle_id VARCHAR NOT NULL,
    timestamp TIMESTAMP NOT NULL,
    latitude DOUBLE PRECISION NOT NULL,
    longitude DOUBLE PRECISION NOT NULL,
    heading INTEGER,
    camera_id VARCHAR,
    image_url TEXT NOT NULL,
    blurred BOOLEAN DEFAULT FALSE,
    processed BOOLEAN DEFAULT FALSE,
    stream_id UUID,
    location GEOGRAPHY(Point, 4326),

    FOREIGN KEY (stream_id) REFERENCES streams(stream_id)
);

CREATE INDEX idx_images_location ON images USING GIST (location);
CREATE INDEX idx_images_timestamp ON images (timestamp DESC);

2. 🎥 streams table

CREATE TABLE streams (
    stream_id UUID PRIMARY KEY,
    vehicle_id VARCHAR NOT NULL,
    start_time TIMESTAMP NOT NULL,
    end_time TIMESTAMP,
    status VARCHAR CHECK (status IN ('active', 'closed', 'error')),
    stream_url TEXT,
    region_id UUID,

    FOREIGN KEY (region_id) REFERENCES coverage_regions(region_id)
);

3. 🚗 vehicles table

CREATE TABLE vehicles (
    vehicle_id VARCHAR PRIMARY KEY,
    type VARCHAR CHECK (type IN ('car', 'drone', 'static')),
    status VARCHAR CHECK (status IN ('active', 'inactive', 'maintenance')),
    last_seen TIMESTAMP,
    software_ver VARCHAR,
    camera_count INT
);

4. 🌍 coverage_regions table

CREATE TABLE coverage_regions (
    region_id UUID PRIMARY KEY,
    name VARCHAR NOT NULL,
    bbox GEOGRAPHY(POLYGON, 4326),
    last_updated TIMESTAMP
);

CREATE INDEX idx_coverage_bbox ON coverage_regions USING GIST (bbox);

5. ⚙️ telemetry table

CREATE TABLE telemetry (
    vehicle_id VARCHAR NOT NULL,
    timestamp TIMESTAMP NOT NULL,
    latitude DOUBLE PRECISION NOT NULL,
    longitude DOUBLE PRECISION NOT NULL,
    speed_kph FLOAT,
    heading INTEGER,
    altitude FLOAT,
    location GEOGRAPHY(Point, 4326),

    PRIMARY KEY (vehicle_id, timestamp)
);

CREATE INDEX idx_telemetry_location ON telemetry USING GIST (location);
CREATE INDEX idx_telemetry_timestamp ON telemetry (timestamp DESC);

6. 🛠️ image_reports table

CREATE TABLE image_reports (
    report_id UUID PRIMARY KEY,
    image_id UUID NOT NULL,
    reporter_ip VARCHAR,
    reason TEXT NOT NULL,
    status VARCHAR CHECK (status IN ('new', 'reviewing', 'resolved')),
    created_at TIMESTAMP NOT NULL DEFAULT NOW(),
    resolved_by VARCHAR,

    FOREIGN KEY (image_id) REFERENCES images(image_id)
);

7. 🧠 processing_jobs table

CREATE TABLE processing_jobs (
    job_id UUID PRIMARY KEY,
    image_id UUID NOT NULL,
    task VARCHAR CHECK (task IN ('blur_faces', 'stitch', 'tag_location')),
    status VARCHAR CHECK (status IN ('pending', 'in_progress', 'done', 'error')),
    created_at TIMESTAMP DEFAULT NOW(),
    completed_at TIMESTAMP,
    error_msg TEXT,

    FOREIGN KEY (image_id) REFERENCES images(image_id)
);

Microservices and their Interaction(HLD)

Here’s a High-Level Design (HLD) of the Real-Time Google Street View system using a microservices architecture, covering:

Key microservices
Their responsibilities
Interactions between them
Communication protocols
Supporting infrastructure

     +----------------+        +------------------+       +------------------+
     |  Capture App / |        |  Ingestion       |       | Telemetry        |
     |  Vehicle Agent +------->+  Service         +------>+ Service          |
     +----------------+        +------------------+       +------------------+
             |                           |
             | Image/Stream              | GPS
             |                           v
             |                   +--------------------+
             |                   | Storage Service     |<-------------+
             |                   +--------------------+              |
             |                           |                            |
             |                           v                            |
             |                   +--------------------+              |
             |                   | Image Processor     |------------>+
             |                   +--------------------+              |
             |                           |                            |
             |                           v                            |
             |                   +--------------------+              |
             |                   | Metadata Indexer    |             |
             |                   +--------------------+              |
             |                           |                            |
             v                           v                            |
     +------------------+        +------------------+         +------------------+
     | Viewer Service   +<-------+ Coverage Service +-------->+ Moderation       |
     +------------------+        +------------------+         +------------------+
             ^
             |
     +------------------+
     |    API Gateway   |
     +------------------+
             ^
             |
     +------------------+
     |      Clients     |
     |  (Web/Mobile)    |
     +------------------+

| Microservice                     | Responsibility                                      |
| -------------------------------- | --------------------------------------------------- |
| **1. Ingestion Service**         | Accepts image/video data from capture vehicles      |
| **2. Image Processor Service**   | Handles stitching, blurring, and tagging            |
| **3. Storage Service**           | Stores and retrieves raw/processed media            |
| **4. Viewer Service**            | Serves real-time and historical images to end-users |
| **5. Coverage Service**          | Tracks coverage regions and availability            |
| **6. Stream Management Service** | Handles live stream sessions                        |
| **7. Telemetry Service**         | Ingests GPS and sensor data from vehicles           |
| **8. Moderation Service**        | Manages user-generated content reports              |
| **9. Metadata Indexer**          | Indexes geo metadata and provides search            |
| **10. Auth & API Gateway**       | Handles authentication, routing, and rate limiting  |

🧱 Description of Each Microservice

1. 📥 Ingestion Service

Input: Images, metadata, live stream requests from edge devices
Tasks: Validate input, enqueue for processing, write metadata
Tech: REST + Kafka/Redis → async pipeline
Dependencies: Telemetry Service, Storage Service

2. 🖼️ Image Processor Service

Input: Messages from ingestion queue
Tasks: Stitching panoramas, blurring faces/plates, tagging location
Output: Processed image stored in object storage
Tech: Workers (Go/Python), TensorFlow for CV, Celery/RabbitMQ

3. 💾 Storage Service

Responsibility: Manages raw & processed image/video blobs
Backend: S3/GCS with versioning
Supports: Pre-signed URLs, CDN cache headers

4. 🌍 Viewer Service

Tasks: Serves latest images/panoramas per location
APIs: GET /live, GET /history, GET /navigation
Reads from: Metadata indexer, storage service

5. 🗺️ Coverage Service

Responsibility: Real-time region coverage and update frequency
Geo-indexes: Images and vehicle GPS logs
Stores: Region polygons, activity timestamps

6. 📡 Stream Management Service

Responsibility: Manage RTMP/WebRTC sessions, session metadata
Handles: Starting/stopping stream endpoints
Integrates with: Viewer service, ingestion

7. 🚗 Telemetry Service

Input: GPS + heading + altitude + speed from vehicles
Storage: Time-series DB (e.g., TimescaleDB)
Used by: Coverage, Image Processor, Viewer

8. 🛡️ Moderation Service

Handles: Reports from users about privacy/content issues
Exposes: APIs for review workflows and admin actions
Uses: queue for manual review

9. 🔍 Metadata Indexer

Responsibility: Index image metadata for fast geo and time queries
Tech: Elasticsearch + PostGIS
Feeds: Viewer, coverage, history services

10. 🔐 Auth & API Gateway

Responsibilities:
- Token validation (OAuth 2.0, JWT)
- Rate limiting, request routing
- Authentication for stream producers and viewers
Tech: NGINX, Kong, Envoy, Istio (for service mesh)

| Type            | Tech Used       | Services                                   |
| --------------- | --------------- | ------------------------------------------ |
| Sync APIs       | REST/gRPC       | API Gateway ↔ Viewer, Coverage, Moderation |
| Async Messaging | Kafka, RabbitMQ | Ingestion → Processing, Indexing           |
| Blob Storage    | S3/GCS          | Images, Streams                            |
| Geo Queries     | PostGIS         | Viewer, Coverage                           |
| Search          | Elasticsearch   | Metadata Indexer                           |

🧰 Supporting Infrastructure

Service Discovery: Consul / Kubernetes DNS
Monitoring: Prometheus + Grafana, ELK
Logging: Fluentd / Loki
CDN: Cloudflare / Akamai for low-latency image serving
Caching: Redis for recent lookups (popular regions)

HLD

Shashank’s Substack

Discussion about this post