Loading...

Change Owns to Parent Ofs

Set start and due date...

Xporter

XML

Word

Printable

Details

Type: Enabler
Priority: Must have
Fix Version/s: PI19
Component/s: MID CBF, MID CBF SW
Labels:

ARTs:

Obs Mgt & Controls
Benefit hypothesis:

Hide

The Reports and Verification functionality will enable efficient parameterized testing of Mid CBF via the automated BDD testing framework.

Show
The Reports and Verification functionality will enable efficient parameterized testing of Mid CBF via the automated BDD testing framework.
Acceptance criteria:
Hide

A BDD Test Execution in the PSI (of XTP-17075) results in a single report with sections.

The report is available for browsing without connecting to the PSI server(s)

Existing data for implemented checkpoints appears in the report. Includes:

WB State Count

Talon Status

Histogram

Packet Stream Repair

Pass/Fail status for the test run is fed back to the pytest-bdd.

Test Strategy section in Solution Intent updated to include details

Signal Verification documentation updated

Any implemented tests that use delay models have been refactored to use the test_parameters interface
Show
A BDD Test Execution in the PSI (of XTP-17075) results in a single report with sections. The report is available for browsing without connecting to the PSI server(s) Existing data for implemented checkpoints appears in the report. Includes: WB State Count Talon Status Histogram Packet Stream Repair Pass/Fail status for the test run is fed back to the pytest-bdd. Test Strategy section in Solution Intent updated to include details Signal Verification documentation updated Any implemented tests that use delay models have been refactored to use the test_parameters interface
Feature Points:
4
Initial Size:
4
WSJF:
0
Epic Link:
AA0.5 MID Digital Signal Chain Processing
Agile Teams:

Team_CIPA
Due Sprint:
Sprint 2
Story Point Burn-up:
Overdue:
Outcomes:
Hide

ska-mid-cbf-system-tests and ska-mid-cbf-signal-verification repos refactored to generate a single test report based on the test parameters and BDD feature file. Cleanup of the report still needs to be merged, along with feeding pass/fail status back to the BDD test (CIP-1746).

Added schema validation for commands received over the LMC-CBF interface. Remaining cleanup: CIP-1749.

Fixed CBF and test discrepancies that were non-compliant with the LMC-CBF interface schemas.

CBF refactored to be controllable via subarray level instead of via lower-level components.
Show
ska-mid-cbf-system-tests and ska-mid-cbf-signal-verification repos refactored to generate a single test report based on the test parameters and BDD feature file. Cleanup of the report still needs to be merged, along with feeding pass/fail status back to the BDD test (CIP-1746). Added schema validation for commands received over the LMC-CBF interface. Remaining cleanup: CIP-1749. Fixed CBF and test discrepancies that were non-compliant with the LMC-CBF interface schemas. CBF refactored to be controllable via subarray level instead of via lower-level components.
Resolved PI.Sprint:
19.6

Feature Checklist:

Outcomes Reviewed, Satisfies Acceptance Criteria, Accepted by FO

Demos:
- OMC_ART_SystemDemo_3
Requirement Status:

PI24 - UNCOVERED
Labels_MIRO:
Mid.CBF_SW Team_CIPA testing

Description

Update ska-mid-cbf-system-tests and ska-mid-cbf-signal-verification to support report generation based on a set of test parameters and expectations as per the AA0.5 CBF Test Strategy.

Includes:

Implement JSON command interface b/w pytest-bdd and SV based on https://gitlab.com/ska-telescope/ska-mid-cbf-system-tests/-/tree/main/test_parameters, with schema validation
Implement JSON command interface b/w pytest-bdd and MCS for delay models based on https://gitlab.com/ska-telescope/ska-mid-cbf-system-tests/-/blob/main/test_parameters/delay_models/delay_models.json, with schema validation on the pytest-bdd side. Most complicated format of the testing should be able to handle sending multiple delay models at a selected time and with a configurable epoch field.
SV: Collect data based on expectations
Update expectations for implemented checkpoint and basic correlation tests
Generate the report (w/ expectations & data) & upload to gitlab artifacts
Implement interface between pytest-bdd and SV for pass/fail results
Refactor existing BDD tests to use new test parameters/expectations interface (no new tests)
Documentation updates

Draft Test Reports:

\\ds\dfs\SKA\docs\CBFDraftTestReport_stage1.docx
\\ds\dfs\SKA\docs\CBFDraftTestReport_stageFINAL.docx

Attachments

Issue Links

mentioned in: Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...

mentioned on

Commit - CIP-1507: WIP: revised test_parameters in preparation for SP-3348.

(1 mentioned on)

Structure

Activity

People

Assignee:: Avison, Adam

Reporter:: Deriabine, Anton

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Feature Progress

Story Point Burn-up: (100.00%)

Feature Estimate: 4.0

	Issues	Story Points
To Do	0	0.0
In Progress	0	0.0
Complete	15	52.0
Total	15	52.0

Dates

Created:: 21/Apr/23 5:52 PM

Updated:: 15/Apr/24 9:38 AM

Resolved:: 04/Sep/23 7:26 AM

MID CBF Parameterized Reports and Verification