at work we are trying to optimise our build/CI pipeline. We have one proprietary Haskell library which contains our business logic, it has more than 200 Haskell files AND
a good dose of TH which makes is very slow to build. Not only that, but the compilation itself is very memory hungry due to the TH expansion. We cannot split it into more modular
projects, for reasons I won't get into. To make things more fun, we cannot simply bump EC2 instance type, for budget constraints. We currently have 2 Jenkins servers:
* One (which I will call "A") builds each new release of the library when we merge a PR into the remote git master branch. This CI server also uploads it to our private Hackage and can potentially
perform other post builds operations, like creating an RPM and so on.
* One (Which I will call "B") which builds and deploy our webapp, a mixture of Haskell and Ruby. It's a modest EC2 t2.medium machine, which we cannot upgrade.
We are uniformly targeting Centos 7 as our production OS.
We are trying to find a way to get the artifacts produced by "A" and package it up into a Haskell library we could "inject" into "B" in our project's `.stack-work` directories (or globally, if that's an option),
so that "B" never has to build "the big Haskell library" from scratch.
Said that, which is our best option? Ideally we would like to "deceive" stack into thinking "the big Haskell library" can be simply copied as a precompiled package, instead of building it every time a new
version is released and we need to update our webapp Haskell microservices to align to the newly released library.
To be clear: If our Haskell microservices are at version X of the "big Haskell library", the problem is not the build time when the library doesn't change, as Jenkins will keep in the workspace the local ".stack-work"
and therefore the library won't be rebuild. The problem is when the library gets bumped to version Y and we need to cope; in that case each Haskell microservice needs to be updated, which means we need to
painfully rebuild the big Haskell library for each workspace. Not good!
Is the recently-released extensible snapshots a possible solution here?
I hope I made myself clear enough. Please share you war stories! ;)
We have a working solution for this, but it is based on using gradle on top of stack. See https://github.com/prezi/gradle-haskell-plugin and these two small blog posts: http://vigoo.github.io/posts/2015-04-22-gradle-haskell-plugin.html and http://vigoo.github.io/posts/2015-12-22-gradle-haskell-plugin-stack.html if you are interested, and feel free to ask me about details; we are using it every day for more than a year now.
Alfredo Di Napoli <[hidden email]> ezt írta (időpont: 2016. júl. 12., K, 9:09):
On 12 July 2016 at 09:15, Daniel Vigovszky <[hidden email]> wrote:--
thanks a lot for the pointers! I will be honest with you; is not that we are looking forward to introduce yet another tool in our stack (no pun intended!), but more than that I think I can surely steal ideas from your blog posts and the `gradle-haskell-plugin`. Thanks a lot!
I will deserve the right of asking stupid questions on the workflow if I have any ;)
Sure, we built it on gradle because it is used by several other teams for other stacks we have to interact with. The main idea of it is completely gradle-free :) Write me if you have questions.
Alfredo Di Napoli <[hidden email]> ezt írta (időpont: 2016. júl. 12., K, 9:25):
|Free forum by Nabble||Edit this page|