I have a large project with multiple small parts to it and I’m wondering about best practices for setting up such a system in Cruise (or any other CI for that matter).

The whole project is managed in the same mercurial repository, and each part has its own build script. There is a master build script which simply calls into each part’s individual build script. Each part (save one) has its own deployment and test targets as well.

So what do I do here? The way I see it, I can manage the whole project as one large pipeline, which each part being a stage which can be executed in order. The only problem I have with that is when a check in doesn’t actually affect a part, it will be rebuilt anyway.

Other way is to have each part be a separate pipeline. The problem here is managing the logical and artifact dependencies. In addition, it would check out multiple copies of the mercurial repository for no really good reason.

So what do people suggest I do here?

Hey Jeff

I don’t think that modelling each part as a stage is the right way to do things, because ideally you want the builds to occur concurrently. Stages are supposed to model consecutive steps on the path to deployment, not dependencies. If you want to keep your build system simple, you could model each part of your system as a separate job within a single stage. Given enough computers, these jobs will all run concurrently. As you point out though, every part gets built every check-in.

I think that logically the correct thing to do is to have a pipeline for each part. You’re right that it would check out multiple copies of the mercurial repository, but mercurial is pretty fast and space-efficient, and unless your check out is enormous, you should have enough disk space. Cruise can help you manage the logical dependencies: http://studios.thoughtworks.com/cruise-continuous-integration/1.0/help/managing_dependencies.html

You can use Cruise’s artifacts repository to manage artifacts. There’s more information here: http://studios.thoughtworks.com/cruise-continuous-integration/1.0/help/cruise_integration.html

Separate pipleines is probably where I would have gone as well, though I’m not understanding about artifact dependencies between projects. If I have a part B that depends on the artifacts from part A, how will it know where to find them? By specifying dependencies in Cruise I can assume it will build in order (A -> B), but in my current build configuration, B in some ways assumes that it will find its dependencies in a specific place. What’s your recommendation here?

Also, is it possible to specify to Cruise that it should only look in a sub directory for changes before attempting to build? Again, because the whole project is set up in a single repository, each part is a sub directory. If I’m making seperate pipelines for each, I’d rather have it so they don’t build unless something in that specific sub directory changed.