Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1526

Reduce friction to Behaviour Driven Development (BDD) through scripting

Details

    • Enabler
    • Not Assigned
    • PI10
    • None
    • Obs Mgt & Controls
    • Hide

      In order to promote a sound BDD process, we need to make it as friction-less as possible. Having these scripts is a big step towards it.

      I expect to see many more tests, test sets, test results in Jira, that are often updated and kept in synch with what is stored in git repos.

      Show
      In order to promote a sound BDD process, we need to make it as friction-less as possible . Having these scripts is a big step towards it. I expect to see many more tests, test sets, test results in Jira, that are often updated and kept in synch with what is stored in git repos.
    • Hide
      1. one can download 1+ test scenarios from Jira into .feature files; if automation exists, then they can be directly run with pytest-bdd
      2. one can edit an existing test scenario in Jira and update the git representation;
      3. the links between test sets and requirements can only be specified when using Jira (not in the upload script);
      4. one can edit a .feature file in the file system and upload it in Jira; this creates new Jira issues of type test or updates existing ones; the feature file can mention tests that exist already along with not-yet existing ones; 
      5. when uploading tests, tests can be tagged with info related to the git branch from which they are taken and the "fix version" they refer to.
      6. uploading a .feature file might refer to a test set which is created by Jira if missing,  or updated if existing; in any case the tests of the test set need to include the ones that are uploaded)
      7. uploading a test execution updates or creates a Jira issue of type "test execution", with all its links.
      8. when tests, test sets, test execution are uploaded into Jira then I can select requirements and view test coverage reports; and view reports showing me how the test results have changed over time, for a specific test.
      9. one can download a test or a test  set or all the tests that satisfy a Jira filter, into a .feature file that is tagged with the appropriate keys.
      10. For the time being we won't use the concept "Test Plan:".
      11. Ideally, the upload/download scripts would allow also to do a dry-run, and display what info would be done.
      12. A mechanism in Jira exists to remove test executions that are older than N days.

      Optionally, as a bonus, one can specify that the transfer (upload/download) should be in fact a bidirectional sync that considers last modification time.

      Show
      one can download 1+ test scenarios from Jira into .feature files; if automation exists, then they can be directly run with pytest-bdd one can edit an existing test scenario in Jira and update the git representation; the links between test sets and requirements can only be specified when using Jira (not in the upload script); one can edit a .feature file in the file system and upload it in Jira; this creates new Jira issues of type test or updates existing ones; the feature file can mention tests that exist already along with not-yet existing ones;  when uploading tests, tests can be tagged with info related to the git branch from which they are taken and the "fix version" they refer to. uploading a .feature file might refer to a test set which is created by Jira if missing,  or updated if existing; in any case the tests of the test set need to include the ones that are uploaded) uploading a test execution updates or creates a Jira issue of type "test execution", with all its links. when tests, test sets, test execution are uploaded into Jira then I can select requirements and view test coverage reports; and view reports showing me how the test results have changed over time, for a specific test. one can download a test or a test  set or all the tests that satisfy a Jira filter, into a .feature file that is tagged with the appropriate keys. For the time being we won't use the concept "Test Plan:". Ideally, the upload/download scripts would allow also to do a dry-run, and display what info would be done. A mechanism in Jira exists to remove test executions that are older than N days. Optionally, as a bonus, one can specify that the transfer (upload/download) should be in fact a bidirectional sync that considers last modification time.
    • 2
    • 2
    • 10.5
    • Team_KAROO
    • Sprint 5
    • Hide

      Traceability between tests and usage scenarios is improved. TManagement of test coverage is enabled by reports that show Jira XTP tickets for which no tests exist.

      Show
      Traceability between tests and usage scenarios is improved. TManagement of test coverage is enabled by reports that show Jira XTP tickets for which no tests exist.
    • 10.6
    • Demonstrated, Accepted by FO
    • PI22 - UNCOVERED

    • testing
    • SPO-863

    Description

      As discussed in the last TCoP https://confluence.skatelescope.org/display/SE/2021-02-09+-+Testing+CoP+Meeting and is being described in the "BDD testing guide" in the developer portal, we will need to create tools that reduce the friction in creating and managing BDD tests.

      In particular we'd like:

      • for a developer to be able to write the Gherkin test scenario in VSCode, store them into git, develop their automation, and finally upload them into Jira XTP project so that everybody can search/see them in a single location (Jira XTP project);
      • for a tester (such as AIV engineers) to be able to write gherkin tests in Jira/xray, (this can already be done)
      • for a tester to be able to identify a test or a test set, export them into a git repo, somehow manage development of their automation or reuse of existing automation, run them directly or have the CI pipeline run them, manually or automatically upload the test results into Jira/Xray.

      To do so we need scripts to manage interaction with Jira/Xray, and in particular:

      • a script to download tests of a test set, exported as .feature files; each test is labeled with a Jira key. (at the moment this is not much work to do manually: GB: true, but we need to automate whatever is worthwhile doing it, and I think this is easy to automate)
      • a script to upload a test or a .feature file into Jira and update the Jira representation of the test; if the tests do not exist then new Jira issues need to be created, otherwise existing ones need to be updated. (which is master: Jira or git .feature file? GB: I would choose the git representation)
      • a script to upload test executions in a format that is compatible with what can be produced by pytest-bdd and what can be accepted by Xray APIs (see https://docs.getxray.app/display/XRAYCLOUD/REST+API). Such an upload has to link the results of execution of each test with the test issue in Jira. If it is missing it needs to be created, possibly empty.

      Ideally, as a bonus, the scripts would do a synchronization of information, not just an upload or download. Synchronization in the sense that the most recent version wins (for example, instead of uploading a test scenario and blindly overwrite the Jira representation, the script might check the modification time of both representations, identify the most recent one, and make sure that that is the one being replicated in both locations).

      We also need a mechanism that automatically removes test executions that are older than a given number of days. This could be a scritp run by cron or something else. There should be somebody who is responsible for managing such a retention policy.

       

       

      Attachments

        Issue Links

          Structure

            Activity

              People

                g.brajnik Brajnik, Giorgio
                g.brajnik Brajnik, Giorgio
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete55.0
                  Total55.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel