Run ML pipeline on AWS ECS

 In our project we have a ML pipeline which is suppressed to be run just a few times a year. Our stakeholders requested pipeline to be not expensive and simple to execute., that's why we decided to utilise AWS ACS. 

In out pipeline with have only 2 steps. The first is data preprocessing, second is inference. Both are computational heavy and require GPUs. This was an obstacle for us, because we wanted to use ECS without managing underlying EC2 infrastructure.

Fortunately for us, we had opportunity to ask for advice AWS Support. Guys were kind to explain us that ECS service has a feature called Capacity Providers. It is possible to specify EC2 as capacity provider and select instances with GPU to be provided on demand, like on the picture below we we selected EC2 Capacity Provider for our cluster.  We can select desired AMI and instance type.

It is quite important to specify desired capacity. As we do not have streaming tasks and our customer can wait until cluster is provisioned we select desired capacity minimum as zero instances.

If you would like to get more information about capacity provider, please, consider the next links:

Working with GPUs on Amazon ECS

If you need to constrain available GPU usage on ECS node, please take a look on Specifying GPUs in your task definition