May 6, 2015
The Problem
Recently we needed to redirect all Amazon Elastic Load Balancer (ELB) HTTP
traffic to HTTPS. AWS ELB doesn't provide this automatic
redirection as a service. ELB will, however, let you map multiple
ports from the ELB into the auto-scaling cluster of nodes attached
to that ELB.
People usually just point both port 80 & 443 to a webserver that
is configured to redirect traffic through the secure port. The
question of how to configure your webserver for this task is asked over &
over
on the internet. People have to go scrape the config snip off the
internet & put it in their webserver's configuration files. You
might be using a different webserver for your new project than you
used for your last.
Lifting this configuration into place also takes some dev-ops
work (chef, puppet, etc) & testing to make sure it works. If you
have to mix redirect-to-https configuration with your other
configuration for the webserver it takes even more care & testing.
Wouldn't it be nicer to have a microservice for this that redirects
out of the box without any configuration needed?
We could map port 80 (HTTP) to our own fast webserver to do the
job of redirecting to HTTPS (TLS). The requirements are just that
it always redirects to HTTPS & doesn't need configuration to do so
(at least in its default mode).
The Solution
I wrote a Haskell service using the fast webserver
library/server combo of Wai & Warp. It only took about an hour to
write the basic service from start time to ready-for-deployment
time. Working on it for an hour solved a problem for us for the
foreseeable future for forcing HTTPS on AWS ELB. It does the job
well & logs in Apache webserver format. We had it deployed the same
day.
The project is open source & can be found on github.com.
Why Haskell?
Haskell can be a great tool for solving systems/dev-ops
problems. Its performance can compete with other popular natively
compiled systems languages like Go, Rust or even (hand-written)
C.
In addition to great performance, Haskell helps you to
communicate your intent in code with precision. Mistakes are often
caught at compile time instead of runtime. You often hear
Haskellers talk about having their code just work after they write
it & it compiles.
After installing the GHC compiler and the `cabal-install` build
tool, compiling a native executable of the webserver is as simple
as these 3 commands in the project root.
After installation you will have a single binary in $PROJECT/.cabal-sandbox/bin/rdr2tls.
Deployment
What gets installed is a native executable with just a few
dynamic links (because GPL licensing). Since we have a nice
self-contained native executable, we have a multitude of options
for deployment. We could create a Debian package. We could package
things up as an RPM. We could deliver the code as a Docker
container.
We chose to deploy our first run of the project as Docker
container. The first deploy was 200MB (because we based the
deployment on the Ubuntu docker image). This is not a huge image
but we wanted to see if we could shrink that if possible.
What if we could take everything out of the image that wasn't
necessary to running our webserver code? There isn't a whole lot
needed to create a working Docker image from an executable. If you
run ldd <binary> on the native executable you'll see the following.
If we package up just the libraries that are linked, is that
enough? No. It didn't work. Michael Snoyman did some digging around
& found we also need some gconv UTF libraries. I also found we
needed /bin/sh for Docker to be happy. We created a small project for
building a base docker image with these things in place. It's just
a few megabytes!
When we inject our webserver into the base image we get a
complete Docker image for our webserver in less than 20MB. That's
not bad!
Into the Rabbit Hole
We went from nearly 200MB to 20MB. Can we do any better? How
deep does the rabbit hole go? Luckily I had the weekend so I could
really geek out on it.
GHC can be configured with a number of options when it is compiled. We can matrix on the following options:
GHC Version: 7.8 or 7.10 (the last two stable)
GHC Build Flavour: (e.g., quick, perf & perf-llvm)
GHC Integer Library: libgmp-based or 'simple'
LLVM Version: 3.4 or 3.5 (the last two stable)
Split Objects: not recommended in the GHC manual (so we didn't)
In addition to tweaking GHC compiler options while installing
GHC, we can tell GHC to compile the code with different
backends:
GHC Backend: asm or llvm
I used a script to run through and compile all the different
combinations of GHC. I ended up with many, many versions of GHC
installed (11GB of them actually). I wanted to see what difference
it would make in the size of the webserver executable.
After compiling the webserver a couple dozen times we see that
flags & options makes a difference. Sizes for the stripped native
executable ranged from 13879600 bytes (13.88MB) to 5963632 bytes
(5.96MB) depending on options. No doubt there will be performance
trade offs in size vs performance. We are just looking at size for
the moment.
If we add UPX in the
mix, we can further shrink the executable to the range of 3022828
bytes (3.02MB) to 1224368 bytes (1.22MB!).
Our 'scratch' base docker image is 3.67MB (w/o libgmp) and
4.19MB (w/ libgmp) currently. If we add a stripped & compressed
executable weighing in at 1.22MB to 3.67MB we should get something
around 5MB. Not to shabby for a complete running Docker image!
REPOSITORY TAG SIZE rdr2tls 7.8.4-perf_llvm-llvm_3_4-integer_gmp-llvm 7.21MB rdr2tls 7.8.4-perf_llvm-llvm_3_4-integer_gmp-asm 7.11MB rdr2tls 7.8.4-perf_llvm-llvm_3_4-integer_simple-llvm 6.69MB rdr2tls 7.8.4-perf_llvm-llvm_3_4-integer_simple-asm 6.59MB rdr2tls 7.8.4-perf-llvm_3_4-integer_gmp-llvm 5.70MB rdr2tls 7.8.4-perf-llvm_3_5-integer_gmp-asm 5.60MB rdr2tls 7.8.4-perf-llvm_3_4-integer_gmp-asm 5.60MB rdr2tls 7.8.4-perf-llvm_3_4-integer_simple-llvm 5.18MB rdr2tls 7.8.4-perf-llvm_3_5-integer_simple-asm 5.08MB rdr2tls 7.8.4-perf-llvm_3_4-integer_simple-asm 5.08MB haskell-scratch integer-gmp 4.19MB haskell-scratch integer-simple 3.66MB
The 7MB LLVM-backend-compiled version is now pushed to Dockerhub.
Appendix: The Data
Stripped Executable Size (bytes)
Version Build Flavour LLVM Integer Library Backend Size 7.8.4 perf_llvm llvm_3_4 integer_simple llvm 13879600 7.8.4 perf_llvm llvm_3_4 integer_gmp llvm 13875952 7.8.4 perf_llvm llvm_3_4 integer_simple asm 13768888 7.8.4 perf_llvm llvm_3_4 integer_gmp asm 13763704 7.8.4 quick llvm_3_4 integer_gmp llvm 11854264 7.8.4 quick llvm_3_4 integer_simple llvm 11841336 7.8.4 quick llvm_3_4 integer_gmp asm 11640248 7.8.4 quick llvm_3_5 integer_gmp asm 11640248 7.8.4 quick llvm_3_4 integer_simple asm 11624760 7.8.4 quick llvm_3_5 integer_simple asm 11624760 7.8.4 perf llvm_3_4 integer_simple llvm 6570680 7.8.4 perf llvm_3_4 integer_gmp llvm 6568888 7.8.4 perf llvm_3_4 integer_gmp asm 6456632 7.8.4 perf llvm_3_5 integer_gmp asm 6456632 7.8.4 perf llvm_3_4 integer_simple asm 6455864 7.8.4 perf llvm_3_5 integer_simple asm 6455864 7.10.1 perf llvm_3_5 integer_gmp llvm 6267568 7.8.4 perf_llvm llvm_3_5 integer_gmp llvm 6267568 7.8.4 perf_llvm llvm_3_5 integer_simple llvm 6267568 7.10.1 perf_llvm llvm_3_5 integer_gmp llvm 6267568 7.10.1 quick llvm_3_5 integer_gmp llvm 6267568 7.10.1 perf llvm_3_4 integer_gmp llvm 6259376 7.10.1 perf_llvm llvm_3_4 integer_gmp llvm 6259376 7.10.1 perf_llvm llvm_3_4 integer_simple llvm 6259376 7.10.1 quick llvm_3_4 integer_gmp llvm 6259376 7.10.1 perf llvm_3_4 integer_gmp asm 5963632 7.10.1 perf llvm_3_5 integer_gmp asm 5963632 7.8.4 perf_llvm llvm_3_5 integer_gmp asm 5963632 7.8.4 perf_llvm llvm_3_5 integer_simple asm 5963632 7.10.1 perf_llvm llvm_3_4 integer_gmp asm 5963632 7.10.1 perf_llvm llvm_3_4 integer_simple asm 5963632 7.10.1 perf_llvm llvm_3_5 integer_gmp asm 5963632 7.10.1 quick llvm_3_4 integer_gmp asm 5963632 7.10.1 quick llvm_3_5 integer_gmp asm 5963632
Compressed Executable Size (bytes)
Version Build Flavour LLVM Integer Library Backend Size 7.8.4 perf_llvm llvm_3_4 integer_simple llvm 3022828 7.8.4 perf_llvm llvm_3_4 integer_gmp llvm 3022228 7.8.4 perf_llvm llvm_3_4 integer_simple asm 2924580 7.8.4 perf_llvm llvm_3_4 integer_gmp asm 2924084 7.8.4 quick llvm_3_4 integer_gmp llvm 2526344 7.8.4 quick llvm_3_4 integer_simple llvm 2523524 7.8.4 quick llvm_3_4 integer_gmp asm 2415588 7.8.4 quick llvm_3_5 integer_gmp asm 2415588 7.8.4 quick llvm_3_4 integer_simple asm 2412936 7.8.4 quick llvm_3_5 integer_simple asm 2412936 7.8.4 perf llvm_3_4 integer_simple llvm 1516816 7.8.4 perf llvm_3_4 integer_gmp llvm 1513672 7.8.4 perf llvm_3_4 integer_simple asm 1412060 7.8.4 perf llvm_3_5 integer_simple asm 1412060 7.8.4 perf llvm_3_4 integer_gmp asm 1409684 7.8.4 perf llvm_3_5 integer_gmp asm 1409684 7.8.4 perf_llvm llvm_3_5 integer_simple llvm 1339448 7.10.1 perf llvm_3_5 integer_gmp llvm 1339192 7.8.4 perf_llvm llvm_3_5 integer_gmp llvm 1339192 7.10.1 perf_llvm llvm_3_5 integer_gmp llvm 1339192 7.10.1 quick llvm_3_5 integer_gmp llvm 1339192 7.10.1 perf llvm_3_4 integer_gmp llvm 1338580 7.10.1 perf_llvm llvm_3_4 integer_gmp llvm 1338572 7.10.1 quick llvm_3_4 integer_gmp llvm 1338572 7.10.1 perf_llvm llvm_3_4 integer_simple llvm 1338540 7.8.4 perf_llvm llvm_3_5 integer_simple asm 1224440 7.10.1 perf_llvm llvm_3_4 integer_simple asm 1224440 7.10.1 perf llvm_3_4 integer_gmp asm 1224368 7.10.1 perf llvm_3_5 integer_gmp asm 1224368 7.8.4 perf_llvm llvm_3_5 integer_gmp asm 1224368 7.10.1 perf_llvm llvm_3_4 integer_gmp asm 1224368 7.10.1 perf_llvm llvm_3_5 integer_gmp asm 1224368 7.10.1 quick llvm_3_4 integer_gmp asm 1224368 7.10.1 quick llvm_3_5 integer_gmp asm 1224368
GHC Compiler Size
Version Build Flavour LLVM Integer Library Size 7.8.4 quick llvm_3_4 integer_gmp 272M 7.8.4 quick llvm_3_5 integer_gmp 272M 7.8.4 quick llvm_3_4 integer_simple 273M 7.8.4 quick llvm_3_5 integer_simple 273M 7.10.1 quick llvm_3_4 integer_simple 332M 7.10.1 quick llvm_3_5 integer_simple 332M 7.8.4 perf_llvm llvm_3_4 integer_gmp 912M 7.8.4 perf_llvm llvm_3_4 integer_simple 913M 7.8.4 perf llvm_3_4 integer_gmp 927M 7.8.4 perf llvm_3_5 integer_gmp 927M 7.8.4 perf llvm_3_4 integer_simple 928M 7.8.4 perf llvm_3_5 integer_simple 928M 7.10.1 perf llvm_3_4 integer_simple 1.1G 7.10.1 perf llvm_3_5 integer_simple 1.1G 7.10.1 perf_llvm llvm_3_5 integer_simple 1.1G