Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1780

CLONE - Reduce friction to Behaviour Driven Development (BDD) through scripting

Details

    • Enabler
    • Must have
    • PI11
    • None
    • Obs Mgt & Controls
    • Hide

      same as the original enabler: sp-1526

      Show
      same as the original enabler: sp-1526
    • Hide

      I'm listing the A.C. that have not yet been met (wrt https://jira.skatelescope.org/browse/sp-1526).

      1. one can download 1+ test scenarios from Jira into .feature files; if automation exists, then they can be directly run with pytest-bdd
      2. one can edit an existing test scenario in Jira and update the git representation;
      3. the links between test sets and requirements can only be specified when using Jira (not in the upload script);
      4. one can edit a .feature file in the file system and upload it in Jira; this creates new Jira issues of type test or updates existing ones; the feature file can mention tests that exist already along with not-yet existing ones; 
      5. when uploading tests, tests can be tagged with info related to the git branch from which they are taken and the "fix version" they refer to.
      6. uploading a .feature file might refer to a test set which is created by Jira if missing,  or updated if existing; in any case the tests of the test set need to include the ones that are uploaded)
      7. uploading a test execution updates or creates a Jira issue of type "test execution", with all its links.
      8. when tests, test sets, test execution are uploaded into Jira then I can select requirements and view test coverage reports; and view reports showing me how the test results have changed over time, for a specific test.
      9. one can download a test or a test  set or all the tests that satisfy a Jira filter, into a .feature file that is tagged with the appropriate keys.
      10. For the time being we won't use the concept "Test Plan:".
      11. the upload/download scripts would allow also to do a dry-run, and display what would be done.
      12. A script can be used to remove test executions that are older than N days from a reference date (by default, today). Also, possibly, with a dry-run flag.
      13. The items written in one of the last comments in SP-1526, namely:
        1. xtp-pull should also be able to print its output in the stdout (perhaps by specifying "-o -")
        2. xtp-pull and xtp-compare could have a "-v-verbose" flag that would print in stderr whether the thing being downloaded/compared is a testset or a test scenario
        3. xtp-compare could format its output as the unix "diff" command, so that it can be used in shell pipelines (for example to create patch files) 
        4. all three the scripts should handle passwords in the same way (if not specified with the flag -p "..." then it should be asked by the terminal without echoing it)
        5. if any of these scripts encounter a "bad credential error" thrown by Jira the should catch it and reraise it with a message that is clear to users
        6. the manual pages in readthedocs should say also:
          • how to install these scripts
          • examples of the output of xtp-pull and xtp-compare
          • a sentence saying that xtp-pull pulls either a testset or a test scenario, depending on what the key is associated to.

      Optionally, as a bonus, one can specify that the transfer (upload/download) should be in fact a bidirectional sync that considers last modification time.

      Show
      I'm listing the A.C. that have not yet been met (wrt https://jira.skatelescope.org/browse/sp-1526 ). one can download 1+ test scenarios from Jira into .feature files; if automation exists, then they can be directly run with pytest-bdd one can edit an existing test scenario in Jira and update the git representation; the links between test sets and requirements can only be specified when using Jira (not in the upload script); one can edit a .feature file in the file system and upload it in Jira; this creates new Jira issues of type test or updates existing ones; the feature file can mention tests that exist already along with not-yet existing ones;   when uploading tests, tests can be tagged with info related to the git branch from which they are taken and the "fix version" they refer to. uploading a .feature file might refer to a test set which is created by Jira if missing,  or updated if existing; in any case the tests of the test set need to include the ones that are uploaded) uploading a test execution updates or creates a Jira issue of type "test execution", with all its links. when tests, test sets, test execution are uploaded into Jira then I can select requirements and view test coverage reports; and view reports showing me how the test results have changed over time, for a specific test. one can download a test or a test  set or all the tests that satisfy a Jira filter, into a .feature file that is tagged with the appropriate keys. For the time being we won't use the concept "Test Plan:". the upload/download scripts would allow also to do a dry-run, and display what would be done. A script can be used to remove test executions that are older than N days from a reference date (by default, today). Also, possibly, with a dry-run flag. The items written in one of the last comments in SP-1526 , namely: xtp-pull should also be able to print its output in the stdout (perhaps by specifying "-o -") xtp-pull and xtp-compare could have a "-v-verbose" flag that would print in stderr whether the thing being downloaded/compared is a testset or a test scenario xtp-compare could format its output as the unix "diff" command, so that it can be used in shell pipelines (for example to create patch files)  all three the scripts should handle passwords in the same way (if not specified with the flag -p "..." then it should be asked by the terminal without echoing it) if any of these scripts encounter a "bad credential error" thrown by Jira the should catch it and reraise it with a message that is clear to users the manual pages in readthedocs should say also: how to install these scripts examples of the output of xtp-pull and xtp-compare a sentence saying that xtp-pull pulls either a testset or a test scenario, depending on what the key is associated to. Optionally, as a bonus, one can specify that the transfer (upload/download) should be in fact a bidirectional sync that considers last modification time.
    • 1
    • 1
    • 18
    • Team_KAROO
    • Sprint 5
    • Hide

      Traceability between tests and usage scenarios is improved. Management of test coverage is enabled by reports that show Jira XTP tickets for which no tests exist.
      The tags in the feature file and the Jira ticket labels correspond when uploading/downloading.

      xray_upload job in skampi pipeline uses a XTP-XXX script to do the upload test results and not curl.

      skampi pipeline uses a script (xtp-upload-cucumber) in skallop to upload cucumber result files instead of using CURL.

      xtp-pull, xtp-push have --dry-run capability.

      xtp-pull and xtp-compare prints in stdout whether the thing being downloaded/compared is a test set or a test scenario.

      xtp-push outputs the downloaded file path.

      xtp-compare: 

      output is formatted as a unix diff
      prints in stdout when files are the same
      All four scripts:

      handle passwords in the same way.
      printing out a clear message to the user when bad credentials are given.
      Documentation:

      Expanded BDD scripts with examples and install steps.
      Updated the package modules.
      Added section for designs
      Resolved documentation build errors and warnings.
      Updated Python packages in build env
      Fixed Python imports
      Scripts documentation is in Developer Portal: BDD utility scripts
      ska-ser-skallop published in artefact repository. skampi is using the new release - post-deployment/tests/smoke/test_validate_bdd_tests.py.test_validate_bdd_features is using the scripts: see output here.

      Demonstrated in Testing CoP and feedback captured here.

       Addressed testing CoP meeting suggestion to provide the user with a URL that can allow to view the ticket in Jira: see here. 

      Show
      Traceability between tests and usage scenarios is improved. Management of test coverage is enabled by reports that show Jira XTP tickets for which no tests exist. The tags in the feature file and the Jira ticket labels correspond when uploading/downloading. xray_upload job in skampi pipeline uses a XTP-XXX script to do the upload test results and not curl. skampi pipeline uses a script (xtp-upload-cucumber) in skallop to upload cucumber result files instead of using CURL. xtp-pull, xtp-push have --dry-run capability. xtp-pull and xtp-compare prints in stdout whether the thing being downloaded/compared is a test set or a test scenario. xtp-push outputs the downloaded file path. xtp-compare:  output is formatted as a unix diff prints in stdout when files are the same All four scripts: handle passwords in the same way. printing out a clear message to the user when bad credentials are given. Documentation: Expanded BDD scripts with examples and install steps. Updated the package modules. Added section for designs Resolved documentation build errors and warnings. Updated Python packages in build env Fixed Python imports Scripts documentation is in Developer Portal: BDD utility scripts ska-ser-skallop published in artefact repository. skampi is using the new release - post-deployment/tests/smoke/test_validate_bdd_tests.py.test_validate_bdd_features is using the scripts: see output here. Demonstrated in Testing CoP and feedback captured here.  Addressed testing CoP meeting suggestion to provide the user with a URL that can allow to view the ticket in Jira: see here. 
    • 12.1
    • Stories Completed, Integrated, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO

    Description

      Same as the original enabler: SP-1526

       

      Attachments

        Issue Links

          Structure

            Activity

              People

                g.brajnik Brajnik, Giorgio
                g.brajnik Brajnik, Giorgio
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 1.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete76.5
                  Total76.5

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel