Building a Massive Scalable Chat Application with SST, DynamoDB, and AWS Lambda

• Introduction
• High-Level Architecture
• Single Table Design in DynamoDB
• Using AWS WebSocket API
• AWS Lambda and Concurrency
• Implementation Details
  • Setting Up SST

Introduction

Building scalable chat applications involves handling real-time communication, persistent data storage, and efficient resource management. With my team at Crombie, I implemented a similar but larger-scale system to support millions of concurrent users. This blog provides a comprehensive guide to replicating such a system using SST, DynamoDB, and AWS Lambda.

High-Level Architecture

Here’s a high-level overview of the architecture:

1. Frontend: The client communicates with WebSocket endpoints for real-time updates.
2. WebSocket API Gateway: Manages WebSocket connections and routes events to AWS Lambda.
3. AWS Lambda: Processes incoming messages and interacts with DynamoDB.
4. DynamoDB: Stores user sessions, messages, and chat metadata using a single table design.

Single Table Design in DynamoDB

What is Single Table Design?

Single table design consolidates all data models into one table, leveraging DynamoDB’s indexing and partitioning capabilities for scalability and efficiency.

Designing the Schema

Why Use Single Table Design?

1. Efficient Queries: Fetch all messages for a user or chat with a single query.
2. Cost-Effective: Reduces read and write operations by grouping related items together.
3. Scalable: Handles high throughput with optimized access patterns.

Using AWS WebSocket API

The AWS WebSocket API enables real-time communication between clients and the backend. Key components include:

1. Routes: Define actions like $connect, $disconnect, and custom events (e.g., sendMessage).
2. Integration with Lambda: Routes trigger Lambda functions to handle WebSocket events.
3. Connection Management: Use DynamoDB to store active connections for broadcasting messages.

Example Workflow

1. Client connects to WebSocket: Triggers $connect Lambda function.
2. Message sent by client: Triggers sendMessage Lambda function.
3. Broadcast message: Lambda function retrieves active connections from DynamoDB and sends messages.

AWS Lambda and Concurrency

Handling High Traffic

1. Concurrency Limits: Set reserved concurrency for critical Lambda functions to avoid resource exhaustion.
2. Batch Processing: Use DynamoDB Streams and Lambda for batch processing of messages.

Error Handling

• Use dead-letter queues (DLQs) to capture failed events.
• Integrate with CloudWatch Logs to monitor function performance.

Implementation Details

Setting Up the SST Project Begin by setting up the SST CLI, which provides an excellent framework for building serverless applications. Install the CLI using the command:

npm install -g sst

Create a new SST project and navigate to the project directory:

npx sst create my-chat-app
cd my-chat-app

This initializes a new project using the latest version of SST, setting the foundation for our application.

Configuring AWS DynamoDB To handle the storage needs of the chat application, we will use DynamoDB. Define a DynamoDB table for storing messages in sst.config.ts:

import { Table } from "sst/constructs";
export default {
  config(_input) {
    return {
      name: "my-chat-app",
      region: "us-east-1",
    };
  },
  stacks(app) {
    app.stack(function StorageStack({ stack }) {
      const table = new Table(stack, "Messages", {
        fields: {
          pk: "string",
          sk: "string",
        },
        primaryIndex: { partitionKey: "pk", sortKey: "sk" },
      });
    });
  },
};

The pk field serves as the partition key, representing the chat room ID, while the sk field is the sort key, representing either a timestamp or a unique message ID. This schema ensures efficient retrieval of messages within a specific chat room.

Implementing Lambda Functions Sending Messages To enable users to send messages, create a Lambda function that writes messages to the DynamoDB table. Define the function in your sst.config.ts file and grant it permissions to access the table:

import { Function } from "sst/constructs";
export default {
  // ... previous config ...
  stacks(app) {
    app.stack(function ApiStack({ stack }) {
      const sendMessage = new Function(stack, "SendMessage", {
        handler: "functions/sendMessage.main",
        environment: {
          TABLE_NAME: process.env.TABLE_NAME,
        },
        permissions: ["dynamodb:PutItem"],
      });
    });
  },
};

Then, create the function implementation in functions/sendMessage.ts:

import { DynamoDBClient, PutItemCommand } from "@aws-sdk/client-dynamodb";
const db = new DynamoDBClient({});
export async function main(event: any) {
  const body = JSON.parse(event.body);
  const { chatRoomId, userId, message } = body;

  const params = {
    TableName: process.env.TABLE_NAME,
    Item: {
      pk: { S: chatRoomId },
      sk: { S: new Date().toISOString() },
      userId: { S: userId },
      message: { S: message },
    },
  };

  await db.send(new PutItemCommand(params));

  return {
    statusCode: 200,
    body: JSON.stringify({ status: "Message sent" }),
  };
}

This function parses the incoming request, extracts the message details, and writes them to DynamoDB.

Real-Time Messaging with WebSocket To deliver messages in real time, integrate WebSocket. Define a WebSocket API in your sst.config.ts file:

import { WebSocketApi } from "sst/constructs";
export default {
  // ... previous config ...
  stacks(app) {
    app.stack(function WebSocketStack({ stack }) {
      const websocketApi = new WebSocketApi(stack, "WebSocketApi", {
        routes: {
          $connect: "functions/connect.main",
          $default: "functions/default.main",
          $disconnect: "functions/disconnect.main",
        },
      });

      stack.addOutputs({
        WebSocketApiEndpoint: websocketApi.url,
      });
    });
  },
};

Create the $connect route handler in functions/connect.ts:

export async function main(event: any) {
  // Logic for connection setup
  return {
    statusCode: 200,
    body: "Connected",
  };
}

With this setup, users can establish WebSocket connections to receive real-time updates.

Improving Scalability and Performance

To ensure the application scales effectively, consider implementing additional optimizations. DynamoDB’s Global Secondary Indexes (GSIs) can enable efficient queries for messages based on user or timestamp. Additionally, caching frequently accessed data using AWS ElastiCache can significantly reduce database load. For handling traffic spikes, configure API Gateway throttling and connection limits.

Monitoring is crucial for scalability. Tools like AWS X-Ray and CloudWatch can provide deep insights into application performance, allowing you to identify bottlenecks and optimize. Finally, implementing a CI/CD pipeline with GitHub Actions or AWS CodePipeline ensures consistent and reliable deployments.

By following this approach, you can build a robust, scalable chat application capable of supporting both small-scale and large-scale use cases. The combination of SST, AWS DynamoDB, and AWS Lambda ensures a modern, serverless solution optimized for real-time communication.

References

• SST Documentation
• Amazon DynamoDB Developer Guide
• AWS Lambda Developer Guide
• API Gateway WebSocket APIs