RackHD Pipeline Stability and Validation

Objective:  Identify a process for how to determine CI/CD pipelines are "good", potentially looking in to a staged approach like a sandbox and promote after PR has been validated. Leveraging mechanisms such as feature flags, continuous jobs running on the sandbox.

Existing pages, epics, stories and chores related to pipeline stability and validation.

RackHD CI/CD development process (draft)

RAC-4549 - Getting issue details... STATUS

• RAC-4571 - Getting issue details... STATUS
• RAC-4162 - Getting issue details... STATUS
• RAC-4595 - Getting issue details... STATUS
• RAC-4212 - Getting issue details... STATUS

RAC-4503 - Getting issue details... STATUS

• Sandbox:  resources
• Feature Flag mechanism: turn on features in the infrastructure  code (i.e. in on-build-config) and specific jobs but leave off in production jobs until test is vetted.
  • Challenges:  Two parallel sets of tests going on, both PR gates would do write back.  Do we want both to do writeback to Github.
  • Example:  Dependency job, enabled by an environment variable.  A parameter in the job will enable the test/feature at run-time.
  • Difference between feature branch and feature flag
    • Flag: runtime check to enable/disable
    • Branch: requires maintenance to keep in sync with master
• Test for the infrastructure
• Sandbox:  An adhoc system to test changes to the infrastructure
  • Perhaps a little more structured so everything
  • Shanghai pre-test:  run in a sandbox Jenkins against a private github.
• Parallel test system for feature flags.  Selectively enable and disable the higher risk features.
• Key Issue:  Visibility of the tests run in sandbox
• Scalability of the process.  How do outside contributors go through same process?
• Resources:  Consideration of what resources are using in the pre-test (sandbox) jenkins vs. production jenkins
• Microservices:  How do we bring in  resources needed for this new set of code into production/sandbox.

Action Items:

    - Consistency of the Sandbox process:  Document the location of the public Sandboxes that should be used and by whom Former user (Deleted) Amy Mullins

    - Wider visibility of how sandboxes should be used in our current process.

    - identify physical resources available for a parallel environment Amy Mullins

    - a simple example of how the feature flag will work Amy Mullins