Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-2988

Automate builds and tests of Panda and Cheetah on PSS Gitlab runners from the SKA Gitlab repository

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Feature
    • Should have
    • PI18
    • COM PSS SW
    • None
    • Data Processing
    • Hide

      Provides automated building and testing of the PSS software on appropriate hardware, which enables regular testing of PSS code behaviour and integrity.

      Show
      Provides automated building and testing of the PSS software on appropriate hardware, which enables regular testing of PSS code behaviour and integrity.
    • Hide
      • PSS servers have been set up as gitlab runners, using Ansible. 
      • Panda and Cheetah are built and unit tested on PSS servers using the gitlab CI pipelines. 
      • A strategy for when/how to trigger build on these machines has been defined. 

       

      Show
      PSS servers have been set up as gitlab runners, using Ansible.  Panda and Cheetah are built and unit tested on PSS servers using the gitlab CI pipelines.  A strategy for when/how to trigger build on these machines has been defined.   
    • 3.5
    • 3.5
    • 0
    • Team_PSS
    • Sprint 5
    • Hide

      The PSS servers have been set up with gitlab runners. On Quad, we did this with ansible and on Tengu we did it manually because it is in constant use for development work.
      Ansible recipes for PSS servers Tengu (GPU) and Kelvin/Quad (FGPA) have been written. We can currently not check them in to the ska-pss-ci-systems repo, due to system team updates to the K8 runners.
      Here are links to feature branches and MRs for completeness:
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/27
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/26
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/25
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/24
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/23
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/tree/at4-945-add-cuda
      https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/tree/at4-945-add-nvidia-driver

      Panda and Cheetah are built on the PSS servers, and Panda is fully unit tested in CI. For Cheetah, we are still having issues with of some of the unit tests causing a red pipeline and we are therefore not including them in our CICD pipeline just yet. We have reviewed those tests and have actions for each one that is causing an issue.
      Example pipelines running with all jobs on Tengu and Quad
      Cheetah: https://gitlab.com/ska-telescope/pss/ska-pss-cheetah/-/pipelines/875289642
      Panda: https://gitlab.com/ska-telescope/pss/ska-pss-panda/-/pipelines/876310069

      We have defined and implemented our triggering strategy, getting as close as possible to our preferred procedure given the limitations of gitlab. Summary here:
      https://confluence.skatelescope.org/pages/viewpage.action?pageId=218414499

      The PSS Machine Configurations are available here:
      https://confluence.skatelescope.org/pages/viewpage.action?pageId=213077220

      Show
      The PSS servers have been set up with gitlab runners. On Quad, we did this with ansible and on Tengu we did it manually because it is in constant use for development work. Ansible recipes for PSS servers Tengu (GPU) and Kelvin/Quad (FGPA) have been written. We can currently not check them in to the ska-pss-ci-systems repo, due to system team updates to the K8 runners. Here are links to feature branches and MRs for completeness: https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/27 https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/26 https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/25 https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/24 https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/merge_requests/23 https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/tree/at4-945-add-cuda https://gitlab.com/ska-telescope/pss/ska-pss-ci-systems/-/tree/at4-945-add-nvidia-driver Panda and Cheetah are built on the PSS servers, and Panda is fully unit tested in CI. For Cheetah, we are still having issues with of some of the unit tests causing a red pipeline and we are therefore not including them in our CICD pipeline just yet. We have reviewed those tests and have actions for each one that is causing an issue. Example pipelines running with all jobs on Tengu and Quad Cheetah: https://gitlab.com/ska-telescope/pss/ska-pss-cheetah/-/pipelines/875289642 Panda: https://gitlab.com/ska-telescope/pss/ska-pss-panda/-/pipelines/876310069 We have defined and implemented our triggering strategy, getting as close as possible to our preferred procedure given the limitations of gitlab. Summary here: https://confluence.skatelescope.org/pages/viewpage.action?pageId=218414499 The PSS Machine Configurations are available here: https://confluence.skatelescope.org/pages/viewpage.action?pageId=213077220
    • 18.6
    • Stories Completed, Outcomes Reviewed, Satisfies Acceptance Criteria, Accepted by FO

    • PI22 - UNCOVERED

    Description

      • Document which machines currently has which hardware/firmware. Agree the priority order of which machines should be configured with Ansible to be part of the CI pipeline.

      • Using Ansible to configure the PSS servers (as decided above) to be capable of running gitlab runners with an FPGA and a GPU. 

      • Verify building and running unit tests of Panda and Cheetah code on the chosen PSS servers using the gitlab CI pipelines. 
      • Define strategies for triggering builds and implement them

      Build up team expertise on how to include FPGA capabilities in a gitlab runner.

      Attachments

        Issue Links

          mentioned in

          Page Loading...

          Page Loading...

          Structure

            Activity

              People

                A.Noutsos Noutsos, Aristeidis
                L.Levin-Preston Levin-Preston, Lina
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 3.5

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete918.0
                  Total918.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel