How to use private hex dependencies in your docker images

There’s not much support for elixir deployments out of the box with Jenkins and ECS. We wanted to simplify our setup by moving common code to a repository all our projects could include — but with a containerized solution dealing with private github repos posed unique challenges. Here is how we solved it:

1. Creating a hex package

This turned out to be very easy. We just created a new mix project, copied the files, renamed the modules, and pushed the code to a GitHub repository.

mix new util
cd util
# mv util.ex lib/ # copy the code and rename the modules
git add . && git commit -m “init”
hub create tripping/util
git push origin master -u
git tag — annotate — message v0.1.0 v0.1.0
git push origin — tags

2. Adding the dependency to our project

Take 1

Hex makes adding dependencies very easy, so all we had to do was add the following line in our deps and run `mix deps.get`

def deps do
  [
    # ...
    {:exutils, github: "tripping/exutils.git", tag: "v0.1.0"},
    # ...
  ]
end

However, as it turns out the github option works only for public repos

Take 2

Well, the alternative for including private repos was actually very simple too, we just had to use the full git url.

def deps do
  [
    # ...
    {:exutils, git: "git@github.com:tripping/exutils.git", tag: "v0.1.0"},
    # ...
  ]
end

This worked great in development, However it failed on deployment.

Our hosting setup

We use AWS ECS to host our microservices, and each of them is built as a docker image which is then started on an ECS cluster. These images are built on Jenkins and are pushed to AWS ECR from which our ECS clusters pull and spin them up.

Our Dockerfile

FROM ${DOCKER_IMAGE}
ENV REFRESHED_AT=2018-03-20 \
    # Set this so that CTRL+G works properly
    TERM=xterm \
    MIX_ENV=${MIX_ENV} \
    PORT=${PORT}
RUN apt-get update \
  && update-ca-certificates --fresh \
  && rm -rf /var/lib/apt/lists/* \
  && mix local.hex --force \
  && mix local.rebar --force
COPY . /opt/app
WORKDIR /opt/app
RUN mix deps.get --only "${MIX_ENV}" \
  && mix release
CMD [ "/opt/app/bootstrap.sh" ]

Docker image build script

# ....
docker build \
  --file "$DOCKERFILE" \
  --tag "${IMAGE_URL}:${IMAGE_VERSION_TAG}" \
  "$ROOT_DIR"
docker push "${IMAGE_URL}:${IMAGE_VERSION_TAG}"
# ....

So, as you can see, we generate an erlang release inside a docker image, as part of the image creation process. Once we build a new image, we just update the task definition to use it.

Well that was a long digression, But the problem with this setup is that since `mix deps.get` runs from inside a docker build process (while the image is being built) there is no way to forward our ssh-agent to authenticate with GitHub.

Take 3

The final solution actually turned out to be simpler than we anticipated (Kudos to the Elixir team for thinking through every possibility). We ended up adding this dependency as a git submodule and using the `path` option in deps.

git submodule add git@github.com:tripping/exutils.git private_deps/exutils
git submodule update — init — recursive
def deps do
  [
    # ...
    {:exutils, path: "./private_deps/exutils/"},
    # ...
  ]
end

Now our jenkins server just updates the submodules before running our build and all our private deps are in place to be built inside docker.

Thanks to Richard Duarte for helping me with this blog post.