Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4030

TMC Test Scenarios and Harness

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Obs Mgt & Controls
    • Hide

      At the moment the code of the test harness is not very modular and easy to extend and to understand; the code is also duplicated in several places for the different integration testing initiatives. For example, the current mechanism for checking what events are raised by the SUT is the one designed for unit testing, that has different requirements than integration testing, which makes it not the most suitable one (eg no timeouts can be specified). Fixing these things will improve uptake by other teams, avoiding that they are forced to reinvent the wheel, and improve their productivity and effectiveness.

      In addition, tests can only be run in a very unflexible way (all together, via 2 steps that are not separated: setting up the SUT and  running  the tests): this slows down test execution and reduces flexibility of the tester (who cannot run a single test at the time, for example). Fixing this will improve speed of execution of tests, and flexibility of how tests are run. as an effect, testers will be more effective in tracing and debugging individual tests, leading to shorter cycle time for tests.

      Finally, when running tests there is little support by logging in terms of tracing what is happening. For example, no clear print out is produced identifying the version of the different SUT components. Fixing this will improve observability of test executions, and as a consequence productivity of testers.

      Show
      At the moment the code of the test harness is not very modular and easy to extend and to understand; the code is also duplicated in several places for the different integration testing initiatives. For example, the current mechanism for checking what events are raised by the SUT is the one designed for unit testing, that has different requirements than integration testing, which makes it not the most suitable one (eg no timeouts can be specified). Fixing these things will improve uptake by other teams, avoiding that they are forced to reinvent the wheel, and improve their productivity and effectiveness. In addition, tests can only be run in a very unflexible way (all together, via 2 steps that are not separated: setting up the SUT and  running  the tests): this slows down test execution and reduces flexibility of the tester (who cannot run a single test at the time, for example). Fixing this will improve speed of execution of tests, and flexibility of how tests are run. as an effect, testers will be more effective in tracing and debugging individual tests, leading to shorter cycle time for tests. Finally, when running tests there is little support by logging in terms of tracing what is happening. For example, no clear print out is produced identifying the version of the different SUT components. Fixing this will improve observability of test executions, and as a consequence productivity of testers.
    • 4
    • 4
    • 0
    • Team_OMCPT
    • Sprint 5
    • Hide

      Expected outcomes:

      • people join to the demos (during the next TCoP meeting)
      • TMC teams start using the new stuff
      • in general, testers that implement integration tests are expected to be faster in developing good tests
      Show
      Expected outcomes: people join to the demos (during the next TCoP meeting) TMC teams start using the new stuff in general, testers that implement integration tests are expected to be faster in developing good tests
    • 22.6
    • Outcomes Reviewed, NFRS met, Satisfies Acceptance Criteria, Accepted by FO
    • PI23 - UNCOVERED

    • Team_OMCPT

    Description

      Enabler to being to define PI22 development needed to enhance the TMC Test Scenarios and Harness.

      We need to revise the current implementation of the test harness used for the tests TMC+X so that

      • it has additional tools to simplify testing (eg a flexible tango event tracer)
      • it has additional tools to report execution (eg pytest-bdd plugin called pytest-bdd-report)
      • it has additional tools to trace what a test does (eg a flexible tango event logger)
      • it is deployed as a python package 
      • it supports a clear separation of the code that sets up the SUT  from the tests
      • it supports a clean mechanism for setting up the needed tango device emulators.

      We also need to refactor some of the existing tests to demonstrate that the above things are useful.

      Attachments

        Structure

          Activity

            People

              g.brajnik Brajnik, Giorgio
              Adam.Avison Avison, Adam
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Feature Progress

                Story Point Burn-up: (0%)

                Feature Estimate: 4.0

                IssuesStory Points
                To Do00.0
                In Progress   00.0
                Complete20.0
                Total20.0

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Structure Helper Panel