Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3442

Mechanism for making integration tests parametric w.r.t. components and environment

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Services
    • Hide

      In the medium and long term, a critical aspect to be considered is the degree of reuse that we want for tests and for the test harness. For example, we might expect integration tests to be reused as-is across skampi, the ITFs, the PSIs for tests related to the control system. Because of their nature, tests of the signal chain probably are expected to have many more differences. When we introduce performance and reliability tests, the degree of reuse will be even smaller.

      The test reuse issue will become more and more important to get it right as the number of tests grows and as different test execution contexts are discovered.

      Show
      In the medium and long term, a critical aspect to be considered is the degree of reuse that we want for tests and for the test harness. For example, we might expect integration tests to be reused as-is across skampi, the ITFs, the PSIs for tests related to the control system. Because of their nature, tests of the signal chain probably are expected to have many more differences. When we introduce performance and reliability tests, the degree of reuse will be even smaller. The test reuse issue will become more and more important to get it right as the number of tests grows and as different test execution contexts are discovered.
    • Hide

      Scope is system and integration automated tests, both when expressed as gherkin scenarios or not.

      Show
      Scope is system and integration automated tests, both when expressed as gherkin scenarios or not.
    • Inter Program
    • Overdue
    • PI23 - UNCOVERED

    • Team_SYSTEM testing

    Description

      We need to make sure that integration tests (even those used in PI integration testing) are agnostic w.r.t. the specific components that are deployed in their SUT, as well as the test environment.

      For example, for Solution goal 1 for PI18, the relevant tests refer to an SUT that is specific for <goal1, PI18>. Once such a test is implemented and we decide that it delivers what it should, then we might promote it to the appropriate regression testing suite. But in that new context, what is the actual SUT that the test should refer to? For example, that test will be routinely executed day in and day out during future PIs, when the context <goal1, PI18> is not relevant any more, just like the SUT it refers to. In a sense, SUTs morph as PIs evolve and we need to make sure that there is a precise and explicit way in which we associate a test to its relevant SUT.

      One such mechanism could be a clever parameterization of tests such that:

      • they identify the components they need (eg. by a list of generic names such as "TMC, CSP.LMC, LOW CBF, SDP"), but 
      • they are agnostic of what are the actual concrete components that constitute the SUT, and
      • such concrete realisations are injected by the deployment/test execution machinery (some fixture) based on information derived from the execution context specified in some field of a test plan/set or via environmental variables.

      Attachments

        Structure

          Activity

            People

              m.deegan Deegan, Miles
              g.brajnik Brajnik, Giorgio
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Feature Progress

                Story Point Burn-up: (0%)

                Feature Estimate: 0.0

                IssuesStory Points
                To Do00.0
                In Progress   00.0
                Complete00.0
                Total00.0

                Dates

                  Created:
                  Updated:

                  Structure Helper Panel