Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3600

Investigate migrating CI/CD capabilities from ska-pss-pipeline repository to ska-pss-cheetah repository

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Spike
    • Could have
    • PI20
    • COM PSS SW
    • None
    • Data Processing
    • Hide

      Faster execution of PSS product tests and consolidation of build/testing capabilities into a single set of CI pipelines, resulting in the need to maintain fewer repositories.  

      Show
      Faster execution of PSS product tests and consolidation of build/testing capabilities into a single set of CI pipelines, resulting in the need to maintain fewer repositories.  
    • Hide

      The capability to run PSS product tests in the CI pipeline of the ska-pss-cheetah repository

       

      Show
      The capability to run PSS product tests in the CI pipeline of the ska-pss-cheetah repository  
    • 1.5
    • 1.5
    • 0
    • Team_PSS
    • Sprint 3
    • Hide

      We have product tests running in the CI pipeline of ska-pss-cheetah. These tests configure and launch a cheetah pipeline using ProTest and where applicable, test the output data products are sensible. We have enabled Product Testing stage in the CI that comes after the Unit test stage. We have enabled these on three different build types of cheetah, each using different cmake configs. Different tests are running on different builds. At the moment, the tests which run the SPS pipeline are failing due to a known bug in cheetah. 

      Cheetah developers can straightforwardly add product testing to other builds should they need to. 
      A screenshot of the implemented product test stage is attached, and the current state of the pipeline can always be viewed in gitlab: https://gitlab.com/ska-telescope/pss/ska-pss-cheetah/-/pipelines

      A write-up of how we are planning to organise the Cheetah CI/CD process can be found here: https://docs.google.com/document/d/1dRDWvTHsyvpcIAU8i6CvwHF2q0VDNTSa/edit?usp=sharing&ouid=115341931076280156800&rtpof=true&sd=true

      Show
      We have product tests running in the CI pipeline of ska-pss-cheetah. These tests configure and launch a cheetah pipeline using ProTest and where applicable, test the output data products are sensible. We have enabled Product Testing stage in the CI that comes after the Unit test stage. We have enabled these on three different build types of cheetah, each using different cmake configs. Different tests are running on different builds. At the moment, the tests which run the SPS pipeline are failing due to a known bug in cheetah.  Cheetah developers can straightforwardly add product testing to other builds should they need to.  A screenshot of the implemented product test stage is attached, and the current state of the pipeline can always be viewed in gitlab: https://gitlab.com/ska-telescope/pss/ska-pss-cheetah/-/pipelines A write-up of how we are planning to organise the Cheetah CI/CD process can be found here: https://docs.google.com/document/d/1dRDWvTHsyvpcIAU8i6CvwHF2q0VDNTSa/edit?usp=sharing&ouid=115341931076280156800&rtpof=true&sd=true
    • 20.4
    • Stories Completed, Outcomes Reviewed, Accepted by FO

    Description

      Currently product testing takes place in the CI pipeline of the ska-pss-pipeline repository which builds cheetah into containers. However, runtime of the product tests can be improved by executing them on a runner on which a cache of the required test vectors is managed locally.

       

      In this feature we will create a prototype product testing stage in the CI/CD pipeline of the ska-pss-cheetah repository, in which building and unit testing of cheetah are already implemented.

      Attachments

        Structure

          Activity

            People

              A.Noutsos Noutsos, Aristeidis
              L.Levin-Preston Levin-Preston, Lina
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Feature Progress

                Story Point Burn-up: (100.00%)

                Feature Estimate: 1.5

                IssuesStory Points
                To Do00.0
                In Progress   00.0
                Complete813.0
                Total813.0

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Structure Helper Panel