Deploying the Winds API to AWS ECS with Docker Compose

6 min read
Nick P.
Nick P.
Published September 6, 2018 Updated April 26, 2021

Winds is a popular RSS and Podcast application provided by Stream – a service that allows you to build news and activity feeds in hours instead of months. Winds is 100% open-source and the backend is easy to install in a local environment or in the cloud – a task that we will cover in this tutorial. To ensure that you make it through the tutorial, please make sure to complete all of the prerequisites.

Prerequisites 📚

As with any tutorial, there are some requirements that come with it. For this post, you’ll need to ensure that you have the following up and running, and ready to go prior to continuing on. If you decide to skip the requirements, you’ll likely get hung up at one spot – and we don’t want that to happen.

  1. Amazon Web Services (AWS) account with Full Access to ECS and ElastiCache
  2. A fresh clone of Winds from https://github.com/GetStream/Winds
  3. An account with MongoDB Atlas or another MongoDB provider (we recommend MongoDB Atlas)
  4. A free account with Stream
  5. An instance of AWS ElastiCache setup and running Redis (copy the URI as you’ll need it shortly)
  6. A free API key from Mercury (this handles RSS article parsing, so it’s very important)
  7. A free set of credentials from Algolia
  8. AWS CLI installed on your machine
  9. ECS CLI installed in addition to AWS CLI
  10. An account on Docker Hub (you can use another provider if you like; however, I highly recommend sticking with Docker Hub)

One additional thing that I’d like to mention is that you should have the following permissions (or similar) on your AWS account:

  • SecretsManagerReadWrite

  • IAMFullAccess

  • AmazonEC2ContainerRegistryFullAccess

  • AmazonECS_FullAccess

Setting up Dependencies 🛠

As we provided an exhaustive list above, hopefully, you have had a chance to go through the various steps and copy your third-party URIs and credentials to move forward. The next step requires that we modify the docker-compose-aws file located in the /api directory of Winds. The file will look like this when we start: https://gist.github.com/astrotars/d038949a2501e91707c6484e64c8cd23 Fill in the credentials as instructed in the docker-compose-aws.yml file. Don’t forget a random value for your JWT. You should end up with a file that looks something like this: https://gist.github.com/astrotars/697aaf9fbcca1afd02a453dcff5e77e4 Note: We’re using the docker-compose-aws.yml file over a docker-compose.yml file because we have two docker-compose files inside of the same directory. By appending “-aws” to the file, we can easily specify what file want to hit when building the environment.

Getting up and Running with ECS CLI 🤔

The Elastic Container Service command line interface by Amazon Web Services (AWS CLI) provides high-level commands to simplify creating, updating, and monitoring clusters and tasks from a local development environment. What’s important here is that the ECS CLI supports Docker Compose files, which is what we’ve used to define how our application can and should run in the cloud. While it’s meant for multi-container applications (we have a file for this as well in docker-compose-aws.yml), we’re going to be using a single container application for the purpose of this tutorial. Let’s go ahead and configure the AWS ECS CLI so that we can get up and running. First, we’ll create a “profile” using the following command: https://gist.github.com/astrotars/04a97308864af89c872e14e058c44c7a Next, we'll complete the configuration with the following command: https://gist.github.com/astrotars/e997265731374cc23af127037c6f64dc Note: Substitute launch type with the launch type you want to use by default (EC2) region_name with your desired AWS region, cluster_name (WINDS) with the name of an existing Amazon ECS cluster or a new cluster to use, and configuration_name (WINDS) for the name you'd like to give this configuration.

Creating a Cluster with an EC2 Task ✍

AWS ECS needs permissions so that your EC2 task can store logs in CloudWatch. This permission is covered by the task execution IAM role. For that, we’ll need to create a task execution IAM role using the AWS CLI. 1. Create a file named task-execution-assume-role.json with the following contents: https://gist.github.com/astrotars/89796ee876d8d7f5db8b5e7bdd1a38ed 2. Create the task execution role (in the same directory as task-execution-assume-role.json): https://gist.github.com/astrotars/be2991706531b70be550c06eb2ac47c9 3. Attach the task execution role policy: https://gist.github.com/astrotars/395c85b32901b0e379204fc714de90db

Create a Cluster and Security Group 🔑

Next, we’ll create an Amazon ECS cluster with security groups. 1. We’ve specified EC2 as the default launch type in the cluster configuration, so the following command creates an empty cluster and a VPC configured with two public subnets: https://gist.github.com/astrotars/eae78f0c0b9ae758861e5b9e082fc513 Note: This command may take a few minutes to complete while resources are created. You will also need to take note of the VPC and Subnet IDs that are created – we’ll be using those shortly. 2. Using the AWS CLI, create a security group using the VPC value from the output in the previous command: https://gist.github.com/astrotars/6312f348487a4d5008c56d9d508ff457 3. Using AWS CLI, we’re going to add a security group rule to allow inbound access on port 80: https://gist.github.com/astrotars/e1aa92cb0e1bd1bad195fa831e59b560

Specify Parameters for AWS ECS 🖥

In addition to the docker-compose-aws.yml file that we’ve created for you, you’ll need to create an ecs-params.yml file with the following contents: https://gist.github.com/astrotars/22e2d174ebaf4bfc630c8d0e8f5f919d Note: This params file is specific to AWS ECS and is required if you want to run the Winds API on AWS. The values you need to specify can be found in your previous requests from above.

Deploying Image to Docker Hub 🚴

In this section, we’ll outline how to build, tag, and upload the Winds API to Docker and AWS.

Creating and Uploading

For this step, you’ll need to login to Docker with the following command: https://gist.github.com/astrotars/8f760f688ec5933bdaeefbf42a935a91 Then, run the following command to build the Docker image (you must be inside of the /api directory): https://gist.github.com/astrotars/4bf3bcf51de67e864308a7ef5588ed5a

Tagging and Pushing

To start, you will need to get the Docker Image ID, you can run docker image list and it will output all of your Docker images. Grab the ID from the one tagged as “winds” and drop it in the command below. The command you’ll need to properly tag the image is: https://gist.github.com/astrotars/5fa70c2825498f0c23f19f7253080421 Now, it’s time to push the tagged image to AWS. You can do that with the following: https://gist.github.com/astrotars/f8b573823ff9a8d42a36ba9a110081bc It will take around 30 seconds but when it’s complete.

Deploying to a Cluster 📤

Now that we have our files and infrastructure configured, we can go ahead and deploy the Docker compose file to ECS with the following command: Note: By default, the command looks for files called docker-compose.yml in the current directory; because we have two files, we need to specify a different docker compose file with the --file option (or -f for short). https://gist.github.com/astrotars/26fa726a37f9a70b64c641c162cffb61 If all went well, you should see the following in your ECS console! If you click on the task, you will notice that there is a public IP address that will allow you to view the API (it should respond with “pong”).

Done! 👏

I hope that you enjoyed this tutorial on how to deploy Winds to AWS using Docker. In future posts, I’ll outline how to do the same deployment but on Google and Digital Ocean. If you’re interested in deploying the frontend, check out this post that outlines how to do so using AWS S3 and CloudFront. Happy coding! 🎉