Stack minimal dependency specification, or dependency tree output

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Stack minimal dependency specification, or dependency tree output

Lyndon Maydwell
Hi Beginners,


I'm finally getting my hands dirty with Stack, and am using it in conjunction with Docker, but not with the built-in docker functionality.

My Dockerfile is constructed so that it first installs a whole bunch of dependencies globally, like so:

...
RUN stack install HUnit
...

Then after that, installs the project:

...
COPY . /app
WORKDIR /app
RUN stack install
...

This means that on repeated docker builds the app build and install time should be limited to just the application itself, because the dependency builds were cached. Which is great! However, I'm currently generating the list of dependencies just by looking at the output of the stack build of the app, and this displays everything as a flat list.

I'd like to see some kind of tree instead, so that when I pre-install the dependencies, I can specify a minimal list, rather than a whole slew of dependencies that would be pulled in transitively anyway.

Is there an easy way to do this?


Regards,

 - Lyndon


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Mark Fine
You can ask stack to build only dependencies and use that as a cache layer for docker. We do something like:

FROM haskell:7.10

WORKDIR /app

COPY LICENSE Setup.hs simple-app.cabal stack.yaml /app/
RUN stack setup
RUN stack build simple-app --only-dependencies

COPY main /app/main
COPY src /app/src
RUN stack build simple-app --copy-bins


On Wed, Feb 24, 2016 at 1:07 PM, Lyndon Maydwell <[hidden email]> wrote:
Hi Beginners,


I'm finally getting my hands dirty with Stack, and am using it in conjunction with Docker, but not with the built-in docker functionality.

My Dockerfile is constructed so that it first installs a whole bunch of dependencies globally, like so:

...
RUN stack install HUnit
...

Then after that, installs the project:

...
COPY . /app
WORKDIR /app
RUN stack install
...

This means that on repeated docker builds the app build and install time should be limited to just the application itself, because the dependency builds were cached. Which is great! However, I'm currently generating the list of dependencies just by looking at the output of the stack build of the app, and this displays everything as a flat list.

I'd like to see some kind of tree instead, so that when I pre-install the dependencies, I can specify a minimal list, rather than a whole slew of dependencies that would be pulled in transitively anyway.

Is there an easy way to do this?


Regards,

 - Lyndon


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Lyndon Maydwell
Hi Mark,


That would be great, and I have tried that, but there is one issue that caused me to take the current approach instead.

The issue is that every change to

* Setup.hs
* simple-app.cabal
* stack.yaml

will cause the docker to consider the copy statement

> COPY LICENSE Setup.hs simple-app.cabal stack.yaml /app/

as a fresh checkpoint, and make the cache unusable. Since I've frequently changing stack.yaml, and app.cabal, this won't help me much.

Not sure if there's a way around that with this method.

Let me know if I've overlooked something with your approach!


 - Lyndon


On Thu, Feb 25, 2016 at 9:04 AM, Mark Fine <[hidden email]> wrote:
You can ask stack to build only dependencies and use that as a cache layer for docker. We do something like:

FROM haskell:7.10

WORKDIR /app

COPY LICENSE Setup.hs simple-app.cabal stack.yaml /app/
RUN stack setup
RUN stack build simple-app --only-dependencies

COPY main /app/main
COPY src /app/src
RUN stack build simple-app --copy-bins


On Wed, Feb 24, 2016 at 1:07 PM, Lyndon Maydwell <[hidden email]> wrote:
Hi Beginners,


I'm finally getting my hands dirty with Stack, and am using it in conjunction with Docker, but not with the built-in docker functionality.

My Dockerfile is constructed so that it first installs a whole bunch of dependencies globally, like so:

...
RUN stack install HUnit
...

Then after that, installs the project:

...
COPY . /app
WORKDIR /app
RUN stack install
...

This means that on repeated docker builds the app build and install time should be limited to just the application itself, because the dependency builds were cached. Which is great! However, I'm currently generating the list of dependencies just by looking at the output of the stack build of the app, and this displays everything as a flat list.

I'd like to see some kind of tree instead, so that when I pre-install the dependencies, I can specify a minimal list, rather than a whole slew of dependencies that would be pulled in transitively anyway.

Is there an easy way to do this?


Regards,

 - Lyndon


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Josh Barney


> On Feb 24, 2016, at 5:10 PM, Lyndon Maydwell <[hidden email]> wrote:
>
> Hi Mark,
>
>
> That would be great, and I have tried that, but there is one issue that caused me to take the current approach instead.
>
> The issue is that every change to
>
> * Setup.hs
> * simple-app.cabal
> * stack.yaml
>
> will cause the docker to consider the copy statement
>
> > COPY LICENSE Setup.hs simple-app.cabal stack.yaml /app/
>
> as a fresh checkpoint, and make the cache unusable. Since I've frequently changing stack.yaml, and app.cabal, this won't help me much.
>
> Not sure if there's a way around that with this method.
>
> Let me know if I've overlooked something with your approach!
>
>
>  - Lyndon
>>

I have found that it works well to use a dummy file that does not change in order to set up a cache. You can then copy over the real file; preserving the cached docker layer.

Josh
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Lyndon Maydwell
Awesome!

I'll give that a go.


Thanks,


  - Lyndon

On Thu, Feb 25, 2016 at 9:28 AM, Josh Barney <[hidden email]> wrote:


> On Feb 24, 2016, at 5:10 PM, Lyndon Maydwell <[hidden email]> wrote:
>
> Hi Mark,
>
>
> That would be great, and I have tried that, but there is one issue that caused me to take the current approach instead.
>
> The issue is that every change to
>
> * Setup.hs
> * simple-app.cabal
> * stack.yaml
>
> will cause the docker to consider the copy statement
>
> > COPY LICENSE Setup.hs simple-app.cabal stack.yaml /app/
>
> as a fresh checkpoint, and make the cache unusable. Since I've frequently changing stack.yaml, and app.cabal, this won't help me much.
>
> Not sure if there's a way around that with this method.
>
> Let me know if I've overlooked something with your approach!
>
>
>  - Lyndon
>>

I have found that it works well to use a dummy file that does not change in order to set up a cache. You can then copy over the real file; preserving the cached docker layer.

Josh
_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners


_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Simon Jakobi
In reply to this post by Lyndon Maydwell
Hi Lyndon,

I'd like to see some kind of tree instead, so that when I pre-install the dependencies, I can specify a minimal list, rather than a whole slew of dependencies that would be pulled in transitively anyway.

There's "stack list-dependencies" but that includes the transitive dependencies.

You can get the dependency tree (or rather dependency graph) with "stack dot --external"

"stack dot --external --depth 0" will show only the direct dependencies of your project.


Cheers,
Simon



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Lyndon Maydwell
Hi Simon,


stack dot --external --depth 0


That's exactly what I was first looking for!

Cheers :)


 - Lyndon

On Thu, Feb 25, 2016 at 9:55 AM, Simon Jakobi <[hidden email]> wrote:
Hi Lyndon,

I'd like to see some kind of tree instead, so that when I pre-install the dependencies, I can specify a minimal list, rather than a whole slew of dependencies that would be pulled in transitively anyway.

There's "stack list-dependencies" but that includes the transitive dependencies.

You can get the dependency tree (or rather dependency graph) with "stack dot --external"

"stack dot --external --depth 0" will show only the direct dependencies of your project.


Cheers,
Simon



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
Reply | Threaded
Open this post in threaded view
|

Re: Stack minimal dependency specification, or dependency tree output

Lyndon Maydwell
Hi again Simon,

While `stack dot --external --depth 0` gets the job done, I had a little play around with the idea of finding the initial nodes on the implicit dependency graph, rather than a traversal from the initial dependency list, and the results for this project are quite a bit shorter:

lyndon@endpin  master ✔ ~/Silverpond/promise_backend stack dot --external | grep -v PromiseBackend | stack-minimal-dependencies-exe
Right (fromList [Node "StringId \"errors\""
                        ,Node "StringId \"snap\""
                        ,Node "StringId \"wreq\""])

lyndon@endpin  master ✔ ~/Silverpond/promise_backend stack dot --external --depth=0
strict digraph deps {
"PromiseBackend" [style=dashed];
"PromiseBackend" -> "aeson";
"PromiseBackend" -> "base";
"PromiseBackend" -> "bytestring";
"PromiseBackend" -> "containers";
"PromiseBackend" -> "errors";
"PromiseBackend" -> "lens";
"PromiseBackend" -> "lens-aeson";
"PromiseBackend" -> "mtl";
"PromiseBackend" -> "snap";
"PromiseBackend" -> "text";
"PromiseBackend" -> "transformers";
"PromiseBackend" -> "unordered-containers";
"PromiseBackend" -> "wreq";
}


Not that it really matters, but thought it might be interesting for anyone who was following this thread.

The code I used to compute this is awful, and lives here: 


:)


Later!

 - Lyndon

On Thu, Feb 25, 2016 at 10:58 AM, Lyndon Maydwell <[hidden email]> wrote:
Hi Simon,


stack dot --external --depth 0


That's exactly what I was first looking for!

Cheers :)


 - Lyndon

On Thu, Feb 25, 2016 at 9:55 AM, Simon Jakobi <[hidden email]> wrote:
Hi Lyndon,

I'd like to see some kind of tree instead, so that when I pre-install the dependencies, I can specify a minimal list, rather than a whole slew of dependencies that would be pulled in transitively anyway.

There's "stack list-dependencies" but that includes the transitive dependencies.

You can get the dependency tree (or rather dependency graph) with "stack dot --external"

"stack dot --external --depth 0" will show only the direct dependencies of your project.


Cheers,
Simon



_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners