AWS Salt Cloud provisioning from Docker

Tweet about this on Twitter16Share on Facebook3Share on Google+0Share on LinkedIn12

SaltStack can be used for different automation purposes. I used it for orchestrating OpenStack and Ceph cluster deployments, later started to use it for Docker container provisioning. Then I discovered really interesting container orchestration tool, Rancher and wanted to automate installation and configuration with SaltStack. There are available repos for Ansible or Puppet, so I wanted to make this with SaltStack as my preferred configuration management tool. Those salt states are added to the official Rancher repo.

In this blog post, I will present interesting design of deploying a Rancher to AWS from locally running Docker container. This is the big picture of how everything looks like:


Creating Docker image

My first challenge was to make salt-cloud working on almost any OS and to make it really easy to use. Here is where Docker comes into place. I decided to make a simple Docker image with installed salt-cloud and awscli tool for managing AWS resources. Also, this is an easy way to control salt-cloud version and to have prepared configuration files instead of doing it manually. Here is salt-cloud Dockerfile:

You can take a look at all the files which are added to this Docker image here. Basically, those files are just salt-cloud configurations and the bash script which replaces AWS key, secret, AMI ID and other related settings. This script also keeps container running in the background:

When started you can check the logs and run additional bash process to have an environment for AWS Salt provisioning:

Deploying Salt master to environment

The second challenge was to deploy completely new Salt master and to add new minions. This took me a lot of time because there were some bugs related to salt-cloud and documentation which are fixed now. Mostly, the users are using existing Salt master and then just provisioning minions to it. However, I wanted to be able to start completely new environment including Salt master. Salt cloud map looks like this:

Two main options here are make_master: True, that tells salt-cloud which VM is a Salt master and salt_interface: private_ips, which means salt-cloud will use private IP addresses to configure minions.

Using GitHub for SaltStack states and pillars

Last part was to use GitHub repo for Salt states and pillars instead of a local Salt file server. When running with Vagrant this is easy, but running on AWS can be difficult. The best option here is to use git file server backend on the Salt master.

Using git as a backend has a lot of benefits, not only when doing provisioning like I described here. How to use git file server backend with Salt in my next post, so stay tuned.

Tweet about this on Twitter16Share on Facebook3Share on Google+0Share on LinkedIn12
Posted in Cloud, Deployment and tagged , , , , .

Alen Komljen

I'm a DevOps/Cloud engineer with experience that spans a broad portfolio of skills, including cloud computing, software deployment, process automation, shell scripting and configuration management, as well as Agile development and Scrum. This allowed me to excel in solving challenges in cloud computing, and the entire IT infrastructure along with my deep interest in OpenStack, Ceph, Docker and the open-source community.