Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4378

Low ITF as a continuous integration platform

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • LOW ART
    • Hide

      The benefits of adopting a continuous integration approach are well known and well documented: "It improves quality, reduces risk, and establishes a fast, reliable, and sustainable development pace." <https://scaledagileframework.com/continuous-integration/>

      Specifically, in the short term the Low ITF Integration Event test efforts are currently impeded by a lack of confidence in the system, due to insufficient testing cadence of a rapidly evolving software system. Automatically running an integration test suite whenever newer versions of subsystems become available will address this. If not addressed, the problem will worsen, as we commence Verification Event testing activities in parallel to continually re-testing new software versions; and again it will worsen when we begin the sequential integration of multiple stations into the AA0.5 array.

      In the medium term, there is an opportunity to increase quality across the project, by providing software teams with rapid feedback on their releases. Also, having a CI platform in place at the Low ITF lays the foundation for us to be able to do low-risk, managed Release-on-Demand into production.

      In the long term, we want to ensure the continuing relevance and value delivered by the Low ITF by providing it with a clear function and purpose (with respect to software) beyond the running of formal integration and verification event test cases.

      Show
      The benefits of adopting a continuous integration approach are well known and well documented: " It improves quality, reduces risk, and establishes a fast, reliable, and sustainable development pace ." < https://scaledagileframework.com/continuous-integration/ > Specifically, in the short term the Low ITF Integration Event test efforts are currently impeded by a lack of confidence in the system, due to insufficient testing cadence of a rapidly evolving software system. Automatically running an integration test suite whenever newer versions of subsystems become available will address this. If not addressed, the problem will worsen, as we commence Verification Event testing activities in parallel to continually re-testing new software versions; and again it will worsen when we begin the sequential integration of multiple stations into the AA0.5 array. In the medium term, there is an opportunity to increase quality across the project, by providing software teams with rapid feedback on their releases. Also, having a CI platform in place at the Low ITF lays the foundation for us to be able to do low-risk, managed Release-on-Demand into production. In the long term, we want to ensure the continuing relevance and value delivered by the Low ITF by providing it with a clear function and purpose (with respect to software) beyond the running of formal integration and verification event test cases.
    • 2
    • 1.5
    • 0
    • Team_VULCAN
    • Sprint 5
    • Hide

      Due to under-estimation of other planned work, we had much less capacity to invest in this feature than we had planned for. Although the KRs of the associated objective (SPO-3395) have all been met, it remains the case that our system-level chart is not subject to regression testing prior to deployment, and this gap continues to impede our integration efforts. This work will therefore need to continue in PI24 (SP-4586).

      Some specific outcomes achieved in PI23 are

      • At the start of the PI, our IE test cases lived in the ska-low-itf-scripts repository, our SFTs lived in the ska-ost-low-aavs3 repository, and our operational notebooks lived in ska-low-jupyter. To take advantage of commonality between these notebooks, we consolidated them all in ska-low-jupyter, which was renamed to ska-low-tests.
      • Common functionality was factored out of the notebooks into an installable aiv-utils package. Unit tests were written for this package, and the notebooks were refactored to use it.
      • A small number of notebook tests were implemented, and the gitlab CI pipeline was updated to run these notebook tests on each commit.
      Show
      Due to under-estimation of other planned work, we had much less capacity to invest in this feature than we had planned for. Although the KRs of the associated objective (SPO-3395) have all been met, it remains the case that our system-level chart is not subject to regression testing prior to deployment, and this gap continues to impede our integration efforts. This work will therefore need to continue in PI24 ( SP-4586 ). Some specific outcomes achieved in PI23 are At the start of the PI, our IE test cases lived in the ska-low-itf-scripts repository, our SFTs lived in the ska-ost-low-aavs3 repository, and our operational notebooks lived in ska-low-jupyter. To take advantage of commonality between these notebooks, we consolidated them all in ska-low-jupyter, which was renamed to ska-low-tests. Common functionality was factored out of the notebooks into an installable aiv-utils package. Unit tests were written for this package, and the notebooks were refactored to use it. A small number of notebook tests were implemented, and the gitlab CI pipeline was updated to run these notebook tests on each commit.
    • 23.6
    • Stories Completed, Outcomes Reviewed, Accepted by FO

    • PI24 - UNCOVERED

    • Team_VULCAN

    Description

      Currently, Low AIV software testing is based on a systems engineering approach where a test is run as part of a formal integration event, and if the test passes then the milestone is achieved and the test may never need to be run again.

      This approach is insufficient for software, which evolves rapidly, and therefore needs to be updated and re-tested on a similarly rapid cadence. In software it is well understood that infrequent releases with large changesets are risky and difficult to integrate; whereas frequent releases with relatively small changesets are much easier to manage. Frequent releases also reduce the turnaround time for bugfixes, which is essential for AIV.

      However, the successful adoption of this higher cadence release process requires continuous integration (CI) tooling to be in place. As much as possible, we need integration and testing of new releases to be automated.

      This is particularly the case at present, because system integration at the ITF is already impeded by a lack of confidence in the software system, because of the rate at which it is evolving, and a lack of automation to support increasing the testing cadence. This situation will only get worse as we begin integration and verification of multiple AA0.5 stations: without some automation in place, we are condemned to test each station manually.

      Our vision is for the Low ITF to be a true CI platform, where new subsystem versions (whether released or gitlab commits) are automatically deployed to the Low ITF cluster, and a suite of integration tests are automatically run to verify the overall (hardware and software) system.

      Attachments

        Structure

          Activity

            People

              Vani.Naram Naram, Vani
              Drew.Devereux Devereux, Drew
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Feature Progress

                Story Point Burn-up: (100.00%)

                Feature Estimate: 2.0

                IssuesStory Points
                To Do00.0
                In Progress   00.0
                Complete412.0
                Total412.0

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Structure Helper Panel