Softwire Blog


Effortlessly add HTTPS to Dokku, with Let’s Encrypt


26 September 2016, by

You’ve written an application deployed using Dokku, and you’ve got it all up and running and great. You’ve heard a lot about why HTTPS is important nowadays, especially the SEO and performance benefits, and you’d like to add all that, with minimal cost and hassle. Let’s get right on that.

Let’s Encrypt is a new certificate authority (an organisation that issues the certificates you need to host an HTTPS site), which provides certificates to sites entirely for free, by totally automating the system, to help get 100% of the web onto HTTPS. Sounds great, right?

To set up this up, you’re going to need to prove you own the domain you’re asking for a certificate for, you need to get the certificates and start using them, and you’re going to want to have some system in place to automatically renew your certificates, so you never have to think about this again (Let’s Encrypt certificates expire every 90 days, to encourage automation). That means we’ve got a few key steps:

  • Generate a key-pair to represent ourselves (you can think of this as our login details for Let’s Encrypt).
  • Complete Let’s Encrypt’s Simple HTTP challenge, by signing and host a given JSON token at /.well-known/acme-challenge/<token> on port 80 for the given domain name, with our public key. This validates the key pair used to sign this token as authorized to issue certificates for this domain.
  • Request a certificate for this domain, signing the request with our now-authorized key.
  • Set up our server to use this key.
  • Set up an automated job to re-request certificates and update the certificate we’re using at regular intervals.

(Interested in the full details of how the Let’s Encrypt process works? Check out their detailed intro: https://letsencrypt.org/how-it-works/)

Fortunately, you barely have to do any of this! In reality, with Dokku, this is pretty much a case of turning a plugin on, and walking away.

How do you actually do this with Dokku?

First up, we need to install dokku-letsencrypt, which will do most of the setup magically for us (see full instructions here). On your server, run the command below for your dokku version (run ‘dokku version’ to check):

# dokku 0.5+
dokku plugin:update letsencrypt

# dokku 0.4
dokku plugin:update letsencrypt dokku-0.4

To configure it, set an email for your app:

dokku config:set --no-restart myapp [email protected]

(This email will receive the renewal warnings for your certificate)

Turn it on:

dokku letsencrypt myapp

For Dokku 0.5+, set up auto-renewal:

dokku letsencrypt:cron-job --add

For Dokku 0.4, you’ll have to manually add a cronjob scheduled every 60 days to kick off the auto-renewal process:

dokku letsencrypt:auto-renew

That’s it!

What just happened?

The magic is in the ‘dokku letsencrypt myapp’ command. When you run that, Dokku-LetsEncrypt:

  • Starts a new service, in a Docker container, and temporarily delegates the top-level .well-known path on port 80 to it in Nginx (the server which sits in front of all your services in Dokku and routes your requests).
  • Generates a key pair, and authorizes it for each of the domains you have configured for your app (see ‘dokku domains myapp’).
  • Uses that key pair to get certificates for each of those domains.
  • Configures Nginx to enable HTTPS with these certificates for each of those domains.
  • Configures Nginx to 301 redirect any vanilla HTTP requests on to HTTPS for each of those domains.

The later auto-renewal steps just do the work to update these certificates later on.

Easy as pie. Don’t forget to check it actually worked though! It’s quite likely you’ll find mixed content warnings when you first turn on HTTPS. Most major CDNs or other services you might be embedding will have HTTPS options available nowadays, so this should just be a matter of find/replacing http: tohttps: throughout your codebase. With that done though, that shiny green padlock should be all yours.

This post originally appeared on Tim Perry’s personal blog.

Improving Open-Source Deployment with Docker


6 April 2016, by

Open-source software has a lot going for it, but easy of use is not typically at the top of the list. Fortunately, it’s rarely a problem; as developers much of the open-source code we use is in simple tools and libraries, and most of the core interactions we have these are managed by package managers, which have focused on building a convenient usable layer to manage this for any project.

That’s not the case for other domains though, especially non-trivial standalone applications. There are a lot of popular open-source tools that follow this model, and require you to install and run them in an environment providing all their core dependencies. WordPress (which runs this blog) is a good example, along with apps like Discourse (a forum we use for internal discussion). These provide great value and they’re great tools, but setup isn’t easy, often involves many manual steps following sometimes painful documentation, and typically then fails because of some inexplicable idiosyncrasy of the server you’re using.

Staytus is another good example of this. Staytus is an open-source web application that provides a status site for your product, aiming to be a beautiful, usable, easy to manage tool that companies can drop into place to give their customers information on how their system is doing. Take a look at their demo site to see it in action.

Staytus, like many other tools in this domain, isn’t effortless to set up though. You have to install the right version of Ruby and all their ruby dependencies (still an annoyingly fiddly process on Windows especially, if you’re not already using Ruby elsewhere), install Node, install, configure and prepare a MySQL server, configure Staytus to glue this all together, and then hook the Staytus startup commands into whatever service running tool you want to use. None of this is cripplingly difficult, but it’s all friction that gets in the way of putting Staytus into users’ hands.

I found Staytus recently, while looking out for exciting new open-source projects, and decided this needed fixing. I wanted to try it out, but all the above was hassle. It would be better if you could run a single command, have this all done for you, and get a server up straight away. I’d also been hankering to have a closer look at Docker, which felt like a great fit for this, so I dived in.

The steps

So, we want to provide a single command you can run which downloads, installs, configures and starts a working Staytus server. To do that, we need a few things:

  • A working Ruby 2.1 environment
  • All the required Ruby dependencies
  • Node.js (for Rails’s JS asset uglification)
  • A configured MySQL server
  • Staytus configuration files, including the MySQL server details
  • A startup script that prepares the database, and starts the service

Automating this with Docker

Docker lets us define an immutable machine image, and provides extremely fast and convenient mechanisms to share, update, and use these images.

In practice you can treat this like an incredibly good virtual machine management system. In reality under the hood the details are quite different – it is providing isolated containers for systems, but through process isolation within a single operating system, rather than totally independent machines – but while those differences power the benefits, they doesn’t really need to affect how you think about the basics of using Docker in practice.

I’m not going to go into the details of Docker in great depth here, I’m just going to look at an example real-world use, at a high-level. If you’re really interested, take a look at their introductory video, or read through their excellent documentation.

What we need to do is define a recipe for an image of a machine that is configured and ready to run Staytus, following the steps above. We can then build this into an actual runnable machine image, and hopefully then just start it to immediately have that machine ping into existence.

Dockerfile

To start with we need the recipe for such a machine. That recipe (the Dockerfile), and the startup script it needs (a simple bash script) are below.

It’s important to note that while this is a very effective & working approach, there are parts of this that are more practical than they are Docker Best Practice. We’ll talk about that later (see ‘Caveats’).

# Start from the standard pre-prepared Ruby image
FROM ruby
MAINTAINER Tim Perry <[email protected]>

USER root

# Run all these commands inside the image
RUN apt-get update && \
    export DEBIAN_FRONTEND=noninteractive && \
    # Set MySQL password to temp-pw - reset to random password later
    echo mysql-server mysql-server/root_password password temp-pw \
      | debconf-set-selections && \
    echo mysql-server mysql-server/root_password_again password temp-pw \
      | debconf-set-selections && \   
    # Install MySQL for data, node as the JS engine for uglifier
    apt-get install -y mysql-server nodejs
    
# Copy the current directory (the Staytus codebase) into the image
COPY . /opt/staytus

# Inside that directory in the image, install our dependencies
RUN cd /opt/staytus && \
    bundle install --deployment --without development:test

# When you run this image, run docker-start.sh
ENTRYPOINT /opt/staytus/docker-start.sh

# Persist the MySQL DB to an external volume
# This means it can be independent of the life of the container
VOLUME /var/lib/mysql

# Persist copies of other relevant files (config, custom themes).
# Contents of this are copied to the relevant places when the container starts
VOLUME /opt/staytus/persisted

EXPOSE 5000

With this saved as Dockerfile inside the root of the Staytus codebase, we can then run docker build . to build (or rebuild) an image following this locally.

An interesting consideration when writing these Dockerfiles is image invalidation. Docker builds intermediate images for each command here, and rebuilding an image only reruns the steps that have been invalidated, using as many from its cache as possible. That means that by writing the Dockerfile as above rebuilding a new image with changes to the Staytus codebase is very cheap; the Ruby, Node and MySQL installation and setup phases are all cached, and we just take that image, copy the new code in, and pull down the dependencies the current codebase specifies. We only rerun the parts from COPY . /opt/staytus down. Small tweaks like this make iterating on your Docker image much easier.

Take a look at this article about working with the Docker build cache if you’re interested in this (and don’t forget to look at Docker’s best practices guide generally)

docker-start.sh

That Dockerfile installs everything required, copies the codebase into the image, and tells Docker to run the ‘docker-start.sh’ script when the image is started as a container.

To actually use this, we need a docker-start.sh script, to manage service startup process. That full content of that is below.

Note that this script includes some further database setup that could have been done above, at image definition time. That’s done here instead, to ensure the DB password is randomized for each container not baked into the published image, so we don’t end up with Staytus images all over the internet running databases with identical default passwords. Docker doesn’t obviate the need for good security practices!

#!/bin/bash
/etc/init.d/mysql start # Start MySQL as a background service

cd /opt/staytus

# Configure DB with random password, if not already configured
if [ ! -f /opt/staytus/persisted/config/database.yml ]; then
  export RANDOM_PASSWORD=`openssl rand -base64 32`

  mysqladmin -u root -ptemp-pw password $RANDOM_PASSWORD
  echo "CREATE DATABASE staytus CHARSET utf8 COLLATE utf8_unicode_ci" | mysql -u root -p$RANDOM_PASSWORD

  cp config/database.example.yml config/database.yml
  sed -i "s/username:.*/username: root/" config/database.yml
  sed -i "s|password:.*|password: $RANDOM_PASSWORD|" config/database.yml

  # Copy the config to persist it, and later copy back on each start, to persist this config 
  # without persisting all of /config (which is mostly app code)
  mkdir /opt/staytus/persisted/config
  cp config/database.yml /opt/staytus/persisted/config/database.yml

  # On the first run only, run the staytus:install task to setup the DB schema.
  bundle exec rake staytus:build staytus:install
else
  # If it's not the first run:

  # Use the previously saved config from the persisted volume
  cp /opt/staytus/persisted/config/database.yml config/database.yml

  # The DB should already be configured. Check if there are any migrations to run though:
  bundle exec rake staytus:build staytus:upgrade
fi

# Start the Staytus service
bundle exec foreman start

Putting this to use

With this written, you can check out the Staytus codebase, run docker build . to build an image of Staytus, and run docker run -d -p 0.0.0.0:80:5000 [built-image-id] to instantly start a container with that image, listening locally on port 80.

For end users, that a lot easier than all the previous setup we had! There’s still a little more we can do though. Having done this, we can publish that image to Docker Hub, and users no longer need to check out the codebase at all.

The full setup now, from a blank slate, is:

  • Install Docker (a single standard platform-specific installer)
  • Run docker run -d -p 0.0.0.0:80:5000 --name=staytus adamcooke/staytus
  • Browse to http://localhost:80

(Note the ‘adamcooke/staytus‘ part; that’s the published image name)

This is drastically easier than following all the original steps by hand, and very hard to do wrong!

I wrote this all up, contributed this back to Staytus itself in July last year, Adam Cooke (the maintainer of Staytus) merged in and published the resulting image, and Staytus is now ready and available for quick easy use. Give it a go!

Caveats

Some is this is not exactly how things should be done in Docker land – there’s more than a few concessions to short-term practicality – but this does work very nicely, and provides exactly the benefits we’re looking for.

The key Docker rule that’s not being follow here is that we’ve put two processes (Staytus and its MySQL server) into a single image, to run as a single container. Instead, we should run two containers (a Staytus container, and a totally standard MySQL container) and link them together, typically using Docker Compose. At the time though Docker Compose wasn’t yet recommended for production use, and to this day moving to this model still makes it a little harder for users to get set up and running that it would be with the one image. There’s ongoing work to finish that up now though, and Staytus is likely to evolve further in that direction soon.