Setting up a build grid#821

Subscribe to Setting up a build grid 3 post(s), 2 voice(s)

 
Avatar Pete Mounce 47 post(s) #2193

So, I’ve enabled my second agent and given it enough resources so it can participate in my build. However, now I’m unsure – my run-unit-test jobs, which are now being attempted in parallel (and therefore maybe by the agent that did not build the code to DLLs), fail because they don’t know where to find the DLLs.

So – how best to proceed?
  • Do I need to define my compiled DLLs from my earlier “BuildCode” stage as artifacts, that are then used by the later stages?
  • Do I need to make the RunUnitTests msbuild target depend-on the msbuild “BuildUnitTests” target – as in, solve this at the msbuild level, but at the cost of each agent compiling all the code?
  • Should I split the pipeline into two; first one to build the code, second one to run the unit tests (but again, where does it get DLLs from; I don’t know how to make a job pull from artifacts?)?
  • Something else?
 
Avatar Chris Turner Moderator 10 post(s) #2201

Hey Pete,

So, there are a few options here.

1) Have both jobs rebuild the DLLs.
2) Have a first stage containing a job that builds the DLLs and publishes them as artifacts, followed by a UnitTest stage that in which the jobs pull down those artifacts before running the UnitTest stage
3) Use dependant pipelines (same workflow as option 2, but without retaining the benefits of having the same pipeline label)

We recommend option 2, as you can be sure that the binary you create right away is the same throughout all steps of your build. That being said, option 1 is easier to get set up. Option 3 doesn’t really sound like it’s what you’re looking for.

As to how to “pull down those artifacts”, we have a restful API to download artifacts. You can read about it here.

We set the environment variables CRUISE_SERVER_URL, CRUISE_PIPELINE_NAME, CRUISE_PIPELINE_LABEL, CRUISE_STAGE_NAME, and CRUISE_JOB_NAME to represent what you are currently running in, so you can create a URL and download the artifact easily. For example, in your case, create a url like $CRUISE_SERVER_URL/files/$CRUISE_PIPELINE_NAME/$CRUISE_PIPELINE_LABEL/DllStage/create-dll-job/something.dll (assuming DllStage/cruise-dll-job is the stage/job that created and uploaded the DLLs). Add an msbuild target to download the DLLs to a location where you can use them. Finally, add a task in your unit test job definitions on the Administration tab to call that target before running your unit test target.

Hope that helps,
Chris

 
Avatar Pete Mounce 47 post(s) #2222

Yes; that helps. Thanks.