Guideline/Best practice of using RackHD PR Gate(Jenkins Test)

Owned by Alan Wei
Last updated: Feb 21, 2017
2 min read

RackHD Developers did a lot of work to make the PR Gate(Jenkins test) more powerful and easy to use, this page shows the guideline and best practices of using PR gate.

Definitions:

Build Trigger: The actions that can trigger one build/test of RackHD PR gate.

There're several triggers now:  

  1.  Any commit change of PR: create a new PR, pushing new commits to PR
  2. Instruction "test this please", any PR comments include the words "test this please" will trigger one build/test of PR gate.

Skip Test: There're times you don't want to trigger the PR gate, for example, create an incomplete  W.I.P PR or a PR with unmet dependency.

Write "[Skip CI]" in the title of PR will prevent the trigger of PR gate.

PR Dependency: RackHD is a multi-repository project, there're times that one PR depends on other PRs of other repository.

In order to embody the dependency information, you must write the instructions in PR description or PR comments in such format "jenkins: depends on ${PR_URL}, ${PR_URL}" .

Guideline:

1.Make sure have written "[Skip CI]" in the title of PR before any trigger action(see definitions/build trigger) if the PR is certain to fail the test. 
   In the other words, make sure a PR is complete when doing  trigger action.

2.Write exact dependency information(see definitions/PR dependency) in PR description or comments.

For example, if PR A depends on B, but B doesn't depends on A and can be tested individually, please don't write B depends on A in PR B.

If A and B are interdependent, please write A depends on B in A and also B depends on A in B.

** If don't follow the guideline, please prepare to do some extra works to offset the consequences. For example:

If A depends on B and B depends on A, when you create PR A, you haven't known the URL of B, then you just created it with no dependency instructions and no "[skip ci]". 
So this is a PR that will certainly fail. And next you create PR B with "depends on A", then also supply "depends on B" in PR A.
Now there will be two problems: FIRST is the creation of PR A triggered a build that certainly fails, this will waste the resources of PR gate; 
SECOND is that the Schedule of jenkins is not certainly time linear, the test that PR A triggered may be finished later than PR B. So the wrong test results may cover the correct one.
If this happens you must use "test this please" to trigger a new build and wait for it finish

Best practice:

  1. A normal single PR: just create it.
  2. An incomplete W.I.P PR: write "[skip ci]" in the beginning of title when creating it.
  3. A PR that has nothing to do with test frame work: write "[skip ci]" in the beginning of title when creating it, for saving test resources.
  4. A PR depends on others PRs: write the instructions in PR description or PR comments in such format "jenkins: depends on ${PR_URL}, ${PR_URL}...".
  5. **Two or more interdependent PRs: write "[skip ci]" in the beginning of title when creating both of the all PRs, then supply the interdependent information in the all PRs.
     Next delete "[skip ci]" of either PR, then write "test this please" in comments and this will trigger one test build and the results will be written back to all PRs.