Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3168

Create a separate Gitlab repository for the testing framework (protest) that is independent of the ska-pss-pipeline repository

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Feature
    • Should have
    • PI18
    • COM PSS SW
    • None
    • Data Processing
    • Hide

      Enables separate builds and installs of the testing framework without requiring to check out the Cheetah/Panda repositories

      Show
      Enables separate builds and installs of the testing framework without requiring to check out the Cheetah/Panda repositories
    • Hide

      A plan for how to package protest so that it is available for both bare metal users and CI pipelines. 

      A first-stage implementation of the plan. 

      Show
      A plan for how to package protest so that it is available for both bare metal users and CI pipelines.  A first-stage implementation of the plan. 
    • 3.5
    • 3.5
    • 0
    • Team_PSS
    • Sprint 5
    • Hide
      • ProTest has been moved to its own repository and removed from the pss-pipeline repository
      • A setup/build/release pipeline has been created using setuptools for ProTest which creates a pip installable archive. The archive is hosted in the SKA CAR.
      • Protest can be installed independently of the pss-pipeline codebases (cheetah/panda/etc) using pip. This will place a protest executable in the user's path. 
      • ProTest's own unit tests and the PSS product tests have been separated within the ProTest repo so users of the code will only get the product tests when they pip install. A separate installation pathway for ProTest developers brings all tests and all config files
      • BDD product tests have been demostrated to work (and pass) on an existing cheetah install (on dokimi) using ProTest installed from the new location
      • The BDD product tests which previously ran in the ska-pss-pipeline CI pipeline using the "local" ProTest are now obtained from the new ProTest repo, where they still run and pass. 
      Show
      ProTest has been moved to its own repository and removed from the pss-pipeline repository https://gitlab.com/ska-telescope/pss/ska-pss-pipeline (old location) https://gitlab.com/ska-telescope/pss/ska-pss-protest (new location A setup/build/release pipeline has been created using setuptools for ProTest which creates a pip installable archive. The archive is hosted in the SKA CAR. Protest can be installed independently of the pss-pipeline codebases (cheetah/panda/etc) using pip. This will place a protest executable in the user's path.  ProTest's own unit tests and the PSS product tests have been separated within the ProTest repo so users of the code will only get the product tests when they pip install. A separate installation pathway for ProTest developers brings all tests and all config files BDD product tests have been demostrated to work (and pass) on an existing cheetah install (on dokimi) using ProTest installed from the new location The BDD product tests which previously ran in the ska-pss-pipeline CI pipeline using the "local" ProTest are now obtained from the new ProTest repo, where they still run and pass. 
    • 18.6
    • Stories Completed, Outcomes Reviewed, Satisfies Acceptance Criteria, Accepted by FO
    • PI22 - UNCOVERED

    Description

      • Plan the work with help from the System Team
      • Implement independent installation of ProTest

      Currently Protest resides as a subdirectory of the pss-pipeline repository (whose purpose is to provide a single "superbuild" of cheetah and panda (the PSS piepline) and the BDD tests contained in ProTest are triggered using a Make target, which calls a bash script, which launches the tests via pytest-bdd. However, cheetah (and panda will run directly on servers and will be installable via the server's package manager (apt, yum, etc) rather than via the pss-pipeline repository. Cheetah will also be installed in this way at a test facility where the ProTest (and other) tests will run.

      ProTest therefore, needs to be independently installable. It needs to be used by the pss-pipeline repository, and as a standalone package for anyone who needs to test cheetah on bare metal. 

      This brings several challenges/questions

       

      • How should ProTest be packaged?
        • pip? Will need to make a pip installable package comprising all of the protest libraries and all the end-to-end tests. This will require the user (whether a pipeline or a sysadmin at a test facility) has python3 and pip3 installed.
        • rmp/deb? Will need to make ProTest into a package that can be installed by an OS's package manager. In this case we'll need to make two packages, one for each package manager family (e.g., yum vs apt). 
        • git clone? The user simply clones the repo manually.
      • How should ProTest be launched? Currently this is acheived by a launch script written in bash that sets up a python virtual env, installs the relevant dependencies into it using pip and run the tests (either unit, or BDD) that the user requests. Such a script would need to be shipped with the ProTest package and would  be callable from the command line. 
        • pip? May need to provide a python based launch script instead that is placed into $PATH with the package is installed. 
        • yum/deb? - can easily provide the launch script and include it in $PATH
        • git clone?  Could wrap into cmake and launch the tests with a Make target. 

      We're not sure at this point which method is best or if others we haven't thought of might be better, therefore some advice/agreement with the system team would be useful. As part of the move into it's own repo, ProTest would be able to take advantage of the ska-ci-cd Make targets that are available for linting/packaging/tagging etc. 

       

      In this first instance we would like to, on the  back of advice from the system team, develop a concrete plan to package protest in such a way that is it available for bare metal users and CI pipelines alike. Then implement this plan over the remainder of, and/or the next PI. 

       

      Attachments

        Issue Links

          Structure

            Activity

              People

                A.Noutsos Noutsos, Aristeidis
                L.Levin-Preston Levin-Preston, Lina
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 3.5

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete726.0
                  Total726.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel