Building stack projects as docker images

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Building stack projects as docker images

Haskell - Haskell-Cafe mailing list
I am building Haskell web services using docker. I do this utilising the stack tool within the official Haskell Docker image. Doing this has a number of advantages

- It's predictable - I can control bringing in new versions of stack and ghc.
- It's easy to share - no stack or Haskell installation is required to build the image
- These services can interact with other services in a controlled environment using Docker compose.

One downside is that I doing this naively means waiting for a modules to download every time docker build is run since the stack build command will start from scratch each time. I worked around this by adding two build steps. The first runs a hello world example with the stack.yml and package.yml from my project. The second build builds the intended code. This trick separated the downloading of dependent libraries from building the code and means day to day code changes are quick to build since the first stack build is cached as a docker image layer. Still though, making a small change to a stack yml file means a long wait re-downloading all dependent libraries.

Firstly, is there a better way of doing this? I know stack supports building within docker but when I worked this way I still found projects depended on the system stack and would commonly fail due to stack bugs.

Secondly, my two "stack build" calls where the first builds a hello world program feels a bit hacky. is there a better way to achieve this?

Thanks,
Dan



_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Building stack projects as docker images

tiredpixel
Dear Dan,

On Tue, 2018-09-04 at 12:21 +0100, Daniel Rolls via Haskell-Cafe wrote:
> Secondly, my two "stack build" calls where the first builds a hello
> world program feels a bit hacky. is there a better way to achieve
> this?

I cannot comment on the Stack parts of your questions, since I don't
currently use it. Regarding the Docker parts, however, have you tried
something like

    COPY [ \
        "cabal.config", \
        "*.cabal", \
        "./"]
   
    RUN cabal update \
        && \
    cabal install -j --only-dependencies

prior to your main compilation layer? That would install the
dependencies once, with the layer cache expiring on change to the lib
declarations, without the need for the 'hello world' program you
describe.

In case you don't freeze your dependencies, you can likely remove the
`cabal.config` line.

Peace,
tiredpixel

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building stack projects as docker images

Theodore Lief Gannon
In reply to this post by Haskell - Haskell-Cafe mailing list
We're dealing with this same hassle at Slant. For starters, instead of your hello-world build, you can use:

stack build --dependencies-only

We've gone a few steps farther to make Kubernetes deployment easier. We have a base container for web services with this Dockerfile:

FROM haskell:8.2.1 as buildenv
WORKDIR /app
RUN apt-get update && apt-get -y install make xz-utils libpq-dev
RUN stack upgrade --binary-version 1.6.1
RUN stack update && stack setup --resolver lts-11.22
RUN stack build Cabal --resolver lts-11.22
RUN stack build haskell-src-exts --resolver lts-11.22
RUN stack build lens --resolver lts-11.22
RUN stack build aeson --resolver lts-11.22
RUN stack build http-conduit --resolver lts-11.22
RUN stack build servant-server --resolver lts-11.22

We'll occasionally make a new version of this with a bumped LTS version. When we started there was no official 8.4-base docker image, but since we're using stack it doesn't really matter. The explicit install of stack-1.6.1 is due to an error that occurs when trying to pull indexes with 1.7.1 on the kubernetes build server; there's probably a cleaner solution but we're not using any 1.7.1 features yet. Optimally the order here should be "least likely to change goes first" but we haven't done any real analysis on that.

From here, individual services have a Dockerfile roughly like so:

FROM debian:jessie as runtimeenv
RUN apt-get update && apt-get install -y libgmp-dev && rm -rf /var/lib/apt/lists/*

FROM <private>/haskell-webserver:lts-11.22 as dependencies
# pre-build any additional slow local dependencies
COPY stack.yaml .
COPY package.yaml .
COPY submodules ./submodules
RUN rm -rf ~/.stack/indices/
RUN stack build --dependencies-only

FROM dependencies as build
COPY . .
RUN stack build --ghc-options="-O2" # additional options as desired

FROM runtimeenv
WORKDIR /app
COPY --from=build /app/.stack-work/install/x86_64-linux/lts-11.22/8.2.2/bin/app-name-exe .

CMD ./app-name-exe

This gives us a reasonable build speed and nice lean deploy container.


On Tue, Sep 4, 2018 at 4:22 AM Daniel Rolls via Haskell-Cafe <[hidden email]> wrote:
I am building Haskell web services using docker. I do this utilising the stack tool within the official Haskell Docker image. Doing this has a number of advantages

- It's predictable - I can control bringing in new versions of stack and ghc.
- It's easy to share - no stack or Haskell installation is required to build the image
- These services can interact with other services in a controlled environment using Docker compose.

One downside is that I doing this naively means waiting for a modules to download every time docker build is run since the stack build command will start from scratch each time. I worked around this by adding two build steps. The first runs a hello world example with the stack.yml and package.yml from my project. The second build builds the intended code. This trick separated the downloading of dependent libraries from building the code and means day to day code changes are quick to build since the first stack build is cached as a docker image layer. Still though, making a small change to a stack yml file means a long wait re-downloading all dependent libraries.

Firstly, is there a better way of doing this? I know stack supports building within docker but when I worked this way I still found projects depended on the system stack and would commonly fail due to stack bugs.

Secondly, my two "stack build" calls where the first builds a hello world program feels a bit hacky. is there a better way to achieve this?

Thanks,
Dan


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.