Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1256

move PACT/Tango solution into production

Details

    • Enabler
    • Not Assigned
    • PI8, PI9
    • None
    • Obs Mgt & Controls
    • Hide
      • all the teams can use PACT/Tango to implement their share of integration tests
      • at least in some simple cases

      By doing so 

      • more effective integration tests can be developed 
      • they will be more reliable and faster than equivalent e2e tests
      • teams will be more autonomous in carrying out such testing
      Show
      all the teams can use PACT/Tango to implement their share of integration tests at least in some simple cases By doing so  more effective integration tests can be developed  they will be more reliable and faster than equivalent e2e tests teams will be more autonomous in carrying out such testing
    • Hide

      Acceptance Criteria

      • it is possible to write a consumer test that involves 2 or more producers, and whose execution is based on replicas of all these producers;
      • one or spikes are run on actual SKA consumers & producers (taken from real SKA projects) so that the verification of the pact requires driving the producer in certain states; we need to stress the fact that these states have to be in a certain sequence; e.g. wrt the state transitions https://confluence.skatelescope.org/pages/viewpage.action?pageId=105416556 we want in the same pact to write tests such that not all the paths in the graph are used (eg a test that puts the provider in CONFIGURING without having to put it in IDLE; a test that puts the provider in ABORTING followed by a test that puts it in CONFIGURING).
      • in any case it is necessary to provide ways to drive the provider in a given state, in such a way that the provider reaches a consistent state (this likely means that the provider needs to offer an API/scripts to alllow it).
      • there is a way for running the consumer tests so that the pact file is written (rewritten or appended - as per pactman).
      • the spike(s) end with 1+ test sets based on pact that are run also with the CI of the consumer(s).
      • the spike(s) are such that they involve at least a provider that has 2+ consumers.
      Show
      Acceptance Criteria it is possible to write a consumer test that involves 2 or more producers, and whose execution is based on replicas of all these producers; one or spikes are run on actual SKA consumers & producers (taken from real SKA projects) so that the verification of the pact requires driving the producer in certain states; we need to stress the fact that these states have to be in a certain sequence; e.g. wrt the state transitions https://confluence.skatelescope.org/pages/viewpage.action?pageId=105416556  we want in the same pact to write tests such that not all the paths in the graph are used (eg a test that puts the provider in CONFIGURING without having to put it in IDLE; a test that puts the provider in ABORTING followed by a test that puts it in CONFIGURING). in any case it is necessary to provide ways to drive the provider in a given state, in such a way that the provider reaches a consistent state (this likely means that the provider needs to offer an API/scripts to alllow it). there is a way for running the consumer tests so that the pact file is written (rewritten or appended - as per pactman). the spike(s) end with 1+ test sets based on pact that are run also with the CI of the consumer(s). the spike(s) are such that they involve at least a provider that has 2+ consumers.
    • 4
    • 4
    • 4.5
    • Team_KAROO
    • Sprint 3
    • Hide

      Teams can (more autonomously) use PACT/TANGO testing tool for integration testing of a consumer device against a set of provider devices that enables more effective, more reliable and faster integration tests.

      Show
      Teams can (more autonomously) use PACT/TANGO testing tool for integration testing of a consumer device against a set of provider devices that enables more effective, more reliable and faster integration tests.
    • 9.5
    • Outcomes Reviewed, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO
    • PI22 - UNCOVERED

    • testing
    • SPO-863

    Description

      After Karoo's demo on https://jira.skatelescope.org/browse/SP-629 what we need is to move that PACT/Tango prototype into production. 

      This entails addressing its limitations - or at least some of them - and assessing the resulting solution.

      My current idea is to address these limitations:

      • single device proxy used in consumer testing: we need a way to be able to test a consumer against a set of replicas of producers, and therefore have interactions in a single pact that involve the same consumer but 2+ providers
      • driving the provider when verifying a pact - sequential sequence of state changes: we need to assess the consequences of this limitation in terms of testing and possibly find a better solution (eg adding to an interaction a set of commands to drive the provider in a state)
      • simplifying the mechanism for recording the pact, so that the pact is automatically created when doing the consumer tests.

      Assessing the resulting solution might be achieved by using the tool on a testing pilot project that involves 2+ consumers and 2+ providers such that at least a provider has 2+ consumers (so that we can practice the idea of consumer-driven contract testing) where event-driven code is not tested. The assessment should answer these questions:

      • can we write effective integration tests in this way and avoid having to develop more complex e2e tests?
      • what kind of behaviour of devices can be tested in this way? (ie what kind of test conditions can be expressed?)  
      • what kinds of test conditions cannot be written in this way?
      • what is the suggested good practice for the life cycle of pacts? (eg should they be stored in git? should they be used as reference specification of an API?)

      Collaborations across trains are encouraged, and maybe have this as a shared objective.

       

      Attachments

        Issue Links

          Structure

            Activity

              People

                g.brajnik Brajnik, Giorgio
                g.brajnik Brajnik, Giorgio
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 4.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete510.0
                  Total510.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel