Transcription
e-commerce application architecture on AWS. Such an application should be able to manage product catalog, inventory, orders, customers, shopping cart, etc. In addition, support search and analytics functionality. We will design our application using microservices and orchestrate them using workflows where necessary.
Let us first understand what an e-commerce flow looks like. The e-commerce company has several geographically spread warehouses where it keeps product inventory and from where the ordered items are delivered. So when a customer places an order on the e-commerce application, the order is processed and delivered by coordinating with warehouses and logistics.
An e-commerce system would typically be made up of order management, warehouse management, and logistic systems. However, we will be designing the central order management system here and treat warehouse management and logistics as external systems with which we simply integrate. These systems interact with one another over a messaging service. Warehouse and logistics management are important domains in their own right, and they deserve a separate discussion of their own.
With this background, now we are ready to design our application. Keeping in mind our design philosophy, let's look at our technical building blocks. Our services are made up of microservices and workflows. In addition, we have data stores, messaging bus, search, and analytics. Let's architect these on AWS now.
On AWS, this is what our architecture would look like. There is a lot of information here. We will look at each component one by one. Let's begin with user management. Users are essentially customers who use the e-commerce application on mobile or web and place orders. These users are managed in Amazon Cognito. This makes our user management serverless and easy.
AWS CloudFront is used to distribute our static and dynamic content with low latency. Based on configuration, it routes static requests to an S3 bucket and dynamic requests to an API Gateway. S3 bucket hosts static content like the e-commerce website's HTML, JavaScript, and image files. Our e-commerce website could be a rich client application. The API Gateway acts as a secure single point of access to microservices.
Microservices are modeled as Lambda functions accessible via API Gateway. These could also be implemented differently, for example, as applications deployed in containers managed by say, AWS Fargate. Let's look at individual microservices. Inventory service can return information about available inventory for a product. Cart service can add or remove items from a cart. Order service receives requests related to orders like create a new order, cancel, or return an order. It invokes appropriate AWS Step Functions as a result. Search service can return search results from indexed contents.
Order related workflows are modeled as AWS Step Functions. Step Functions allow you to orchestrate multiple AWS services into serverless workflows. We have New Order, Cancel Order, and Return Order Step Functions. These Step Functions are invoked by an Order service Lambda function. Although we can directly invoke these Step Functions from API Gateway, we have an Order service in front of them so that it can return a meaningful response to the caller after invoking the Step Function.
Let's look at individual Step Functions. This is the New Order AWS Step Function. It is made up of various Lambda functions, SQS, and SNS integrations. At the first step, an order is created in DynamoDB in draft status. Next, order payment is validated. If the payment is invalid, the order is marked with invalid payment status. If the payment is valid, order status is changed to confirmed, and then order shipment request is sent via SNS to warehouse and logistics systems. As a last step, the customer is notified about the current order status.
Here's what the Cancel Order AWS Step Function looks like. As a first step, an order is checked for cancellation eligibility. For example, an order that is already in transit cannot be canceled, and so on. If it is eligible to be canceled, then order status is set to canceled, and then shipment cancellation request is sent via SQS to warehouse and logistics systems. As a last step, the customer is notified about the action taken on the order.
Here's what a Return Order AWS Step Function looks like. As a first step, order's return eligibility is checked. For example, an order may not be eligible for return after a certain number of days post-delivery. If it is eligible for return, then order status is set to return initiated, and next, a reverse shipment request is sent to warehouse and logistics systems so that the product can be picked up from the customer's address and shipped back to the warehouse. As a last step, the customer is notified about the order's current status.
Now that we understand how microservices and Step Functions work, let's look at data storage. Our choice of database here is DynamoDB. DynamoDB is a fast, highly scalable NoSQL database. Having said that, you can choose an RDBMS or a combination of databases for your data storage needs depending on your skill sets and unique requirements.
Key entities we are going to deal with are customers, products, inventory, and orders. Our DynamoDB tables are Product, Inventory, Cart, and Order. Since users or customers are managed by Amazon Cognito, we will not model them separately in our database.
The Product table has category, product ID, product name, and other attributes. Primary key is made up of category as the partition key and product ID as the sort key. In addition, we have a global secondary index which has product ID as primary key. Therefore, on the base table, we can efficiently do queries like get all products for a certain category, get product for a certain category and product ID combination. On the global secondary index, we can do a query like get product for a certain product ID. It is important to note that we have chosen our primary key, partition key, and global secondary index based on our access patterns.
The Inventory table stores product inventory available in each warehouse. It has product ID, warehouse code, and available quantity as attributes. Primary key is made up of product ID as partition key and warehouse code as sort key. On this table, we can efficiently query for the following: get all warehouse codes and quantities for a particular product, get available quantity for a certain product in a particular warehouse.
This is the shopping cart table, and it stores the products and quantity a customer has placed in the cart. Primary key is made up of customer ID as partition key and cart addition timestamp as sort key. Therefore, you can efficiently query this table as follows: get all products and quantities in a particular customer's cart, get all products quantities in a particular customer's cart added in a date range.
This is the Order table, and it stores information about orders placed by customers. Primary key is made up of customer ID as partition key and order creation timestamp as sort key. In addition, it has order ID, status, product ID, quantity, and amount as other attributes. An assumption here is that every order has only one product. Therefore, if a customer is trying to buy two different products, it results in two different orders. Note that there is a global secondary index created using order ID as the primary key.
Let's look at queries you can efficiently execute on the base table: get all orders placed by a specific customer, get all orders placed by a specific customer in a certain date range. On the global secondary index, we can use the following query: get order details for a specific order ID. Remember that it is possible to design the tables differently as well in DynamoDB, for example, everything in just one table or fewer tables, and so on. However, we will go with our current design.
Now that we understand how our data is laid out, it's time to find insights and trends in the data. Let's see how we can collect and analyze large amounts of data efficiently. You can stream DynamoDB data as events into Kinesis Data Streams, from where it is fed into Kinesis Firehose, which can then save the data into S3. From here on, you could use Athena to query the S3 data and visualize it in QuickSight, and also use SageMaker for machine learning with this data. Genesis Firehose can also feed the data into OpenSearch for indexation, which in turn can be queried by the search service.
With this, we come to the end of our discussion. This should give you a good idea of how to architect an e-commerce application on AWS.