Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-2747

Test multi-scan visibility receive in persistent SDP deployment

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Feature
    • Must have
    • PI16
    • COM SDP SW
    • None
    • Data Processing
    • Hide

      Being able to receive and store visibility data from multiple successive scans is a fundamental requirement for SDP at AA0.5. Running regular tests of the multi-scan visibility receive processing script in the persistent SDP deployment is a way to get data on the robustness of the system and help us to uncover any problems that need to be resolved.

      Show
      Being able to receive and store visibility data from multiple successive scans is a fundamental requirement for SDP at AA0.5. Running regular tests of the multi-scan visibility receive processing script in the persistent SDP deployment is a way to get data on the robustness of the system and help us to uncover any problems that need to be resolved.
    • Hide
      • Changes to the SDP and its processing scripts to support multiple scans in visibility receive are complete, and they have been integrated and released.
      • Multi-scan visibility receive test is run as part of the scheduled tests of the persistent SDP deployment.
      • Jupyter notebook to demonstrate how to run multi-scan visibility receive is available in the SDP notebook repository.
      Show
      Changes to the SDP and its processing scripts to support multiple scans in visibility receive are complete, and they have been integrated and released. Multi-scan visibility receive test is run as part of the scheduled tests of the persistent SDP deployment. Jupyter notebook to demonstrate how to run multi-scan visibility receive is available in the SDP notebook repository.
    • Intra Program
    • 2
    • 2
    • 0

    • Use Case USE-22 Operate and Monitor Telescope

    • Component Version REL-273 SDP Local Monitoring and Control v0.20.1

      Component Version REL-272 SDP Local Monitoring and Control v0.20.0

      Sub-system Version REL-191 SDP 0.13.0

      Sub-system Version REL-107 SDP 0.12.0

    • Team_ORCA, Team_YANDA
    • Sprint 4
    • Hide

      The work was demoed at DP ART System Demo 16.4 (slides: https://docs.google.com/presentation/d/14SiYiSfDmsqgO3yZI5zqLqvzEnJKKCKhmPwW-Cnw4EY/edit)

      AC1: The two teams have finalized and merged all outstanding merge requests that were related. SDP 0.12.0 has been released (and then 0.12.1) as well as the latest vis-receive script, 0.6.0, both support running multiple scans. All SDP sub-components have been updated accordingly and their latest versions incorporated in the newest SDP release.

      SDP releases:  REL-107, REL-310

      vis-receive processing script: https://artefact.skao.int/#browse/browse:docker-internal:v2%2Fska-sdp-script-vis-receive%2Ftags%2F0.6.0

      AC2: The two teams have updated the test in ska-sdp-integration to run the multi-scan visibility receive. These updates have automatically been migrated to the scheduled persistent SDP tests. example output: https://gitlab.com/ska-telescope/sdp/ska-sdp-integration/-/jobs/3214828047#L119

      Team ORCA has also implemented a 20-minute long alternating-scans test, which calls the multi-scan visibility receive script. This is set up to run on schedule, once every day, on the persistent SDP. example output: https://gitlab.com/ska-telescope/sdp/ska-sdp-integration/-/pipelines/683341283 (currently on a branch)

      AC3: Team Yanda implemented Jupyter notebook available at:
      https://sdhp.stfc.skao.int/binderhub/v2/gl/ska-telescope%2Fsdp%2Fska-sdp-notebooks/main

      for a description see "demo 2" slides https://docs.google.com/presentation/d/1nE9zAEpH4sJIBQugt7X_RRqwz1uTeSzSQjpzey5wUFs/edit#slide=id.g17a624d614d_0_72

      Show
      The work was demoed at DP ART System Demo 16.4 (slides: https://docs.google.com/presentation/d/14SiYiSfDmsqgO3yZI5zqLqvzEnJKKCKhmPwW-Cnw4EY/edit ) AC1: The two teams have finalized and merged all outstanding merge requests that were related. SDP 0.12.0 has been released (and then 0.12.1) as well as the latest vis-receive script, 0.6.0, both support running multiple scans. All SDP sub-components have been updated accordingly and their latest versions incorporated in the newest SDP release. SDP releases:  REL-107 , REL-310 vis-receive processing script: https://artefact.skao.int/#browse/browse:docker-internal:v2%2Fska-sdp-script-vis-receive%2Ftags%2F0.6.0 AC2: The two teams have updated the test in ska-sdp-integration to run the multi-scan visibility receive. These updates have automatically been migrated to the scheduled persistent SDP tests. example output: https://gitlab.com/ska-telescope/sdp/ska-sdp-integration/-/jobs/3214828047#L119 Team ORCA has also implemented a 20-minute long alternating-scans test, which calls the multi-scan visibility receive script. This is set up to run on schedule, once every day, on the persistent SDP. example output: https://gitlab.com/ska-telescope/sdp/ska-sdp-integration/-/pipelines/683341283 (currently on a branch) AC3: Team Yanda implemented Jupyter notebook available at: https://sdhp.stfc.skao.int/binderhub/v2/gl/ska-telescope%2Fsdp%2Fska-sdp-notebooks/main for a description see "demo 2" slides  https://docs.google.com/presentation/d/1nE9zAEpH4sJIBQugt7X_RRqwz1uTeSzSQjpzey5wUFs/edit#slide=id.g17a624d614d_0_72
    • 16.4
    • Stories Completed, Integrated, BDD Testing Passes (no errors), Outcomes Reviewed, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO

    • PI24 - UNCOVERED

    • DP-SDP: G1 SOL: G1

    Description

      Introduction

      The changes to the SDP control system and processing scripts to support multiple scans in visibility receive were almost complete at the end of PI15. There are merge requests pending in:

      • Telescope model (to implement the new command and attribute schemas);
      • Local monitoring and control;
      • Processing scripts;
      • Scripting library;

      with most of the necessary changes. They need to be finished off, integrated and released.

      The scheduled test of the persistent SDP system should be updated to run a multi-scan version of the visibility receive test. It would be enlightening if this test is a longer version of the standard test, for example, it could execute alternating scans for an extended period.

      You should also create a Jupyter notebook to explain how to run the multi-scan version of the visibility receive processing script.

      Who?

      • SDP developers
      • AIV engineers
      • Commissioning scientists

      What?

      See acceptance criteria.

      Why?

      See benefit hypothesis.

      Attachments

        Issue Links

          Is delivered by

          Sub-system Version - A version of a sub-system, used in the Release Management and Configuration Management process REL-107 SDP 0.12.0

          • Released

          Sub-system Version - A version of a sub-system, used in the Release Management and Configuration Management process REL-191 SDP 0.13.0

          • Released

          Component Version - Component/module/library releases REL-272 SDP Local Monitoring and Control v0.20.0

          • Released

          Component Version - Component/module/library releases REL-273 SDP Local Monitoring and Control v0.20.1

          • Released
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          mentioned on

          GitLab Commit - Merge branch 'sp-2747' into 'master'

          GitLab Commit - Merge branch 'sp-2747-new-sdp-names' into 'master'

          GitLab Commit - SP-2747: fix linting

          GitLab Commit - SP-2747: update default scan duration to 4 seconds

          GitLab Commit - SP-2747: add test id

          GitLab Commit - SP-2747: add additional logging

          GitLab Commit - SP-2747: add back verbosity

          GitLab Commit - SP-2747: add ci tools

          GitLab Commit - SP-2747: add config

          GitLab Commit - SP-2747: add docs and remove axit at fail

          GitLab Commit - SP-2747: add eb id into global

          GitLab Commit - SP-2747: add exit at fail to low

          GitLab Commit - SP-2747: add getting next target id

          GitLab Commit - SP-2747: add jq to ci image

          GitLab Commit - SP-2747: add nettools

          GitLab Commit - SP-2747: add new base image build

          GitLab Commit - SP-2747: add oso scripting test

          GitLab Commit - SP-2747: add post install

          GitLab Commit - SP-2747: add release config

          GitLab Commit - SP-2747: add SDP Leafnode entrypoint

          GitLab Commit - SP-2747: add sdp-storage

          GitLab Commit - SP-2747: add test

          GitLab Commit - SP-2747: add wait before deploying shared

          GitLab Commit - SP-2747: allow for updating observation

          GitLab Commit - SP-2747: apply review suggestion

          GitLab Commit - SP-2747: assign test to jira

          GitLab Commit - SP-2747: change back timeout to 150

          GitLab Commit - SP-2747: change cdm version

          GitLab Commit - SP-2747: Change image name

          GitLab Commit - SP-2747: change timeouts to 150

          GitLab Commit - SP-2747: change to using the py10 image

          GitLab Commit - SP-2747: delete accidental commits

          GitLab Commit - SP-2747: disable oet tests

          GitLab Commit - SP-2747: enable console

          GitLab Commit - SP-2747: explicitly set entrypoint

          GitLab Commit - SP-2747: fix automatic integration test settings

          GitLab Commit - SP-2747: fix generation of unique id

          GitLab Commit - SP-2747: fix import

          GitLab Commit - SP-2747: fix incorrect install

          GitLab Commit - SP-2747: fix incorrect templates

          GitLab Commit - SP-2747: fix missing changes

          GitLab Commit - SP-2747: fix missing entry point logs

          GitLab Commit - SP-2747: fix missing install

          GitLab Commit - SP-2747: fix missing observation

          GitLab Commit - SP-2747: fix missing parts

          GitLab Commit - SP-2747: fix waiting

          GitLab Commit - SP-2747: fix worng json

          GitLab Commit - SP-2747: force test execution

          GitLab Commit - SP-2747: Import reshuffling

          GitLab Commit - SP-2747: incease storage to 20

          GitLab Commit - SP-2747: increase storage

          GitLab Commit - SP-2747: install sdp storage post

          GitLab Commit - SP-2747: make assign resources test suspect

          GitLab Commit - SP-2747: make observation a non class var

          GitLab Commit - SP-2747: mark test as suspect

          GitLab Commit - SP-2747: Merge master

          GitLab Commit - SP-2747: Merge remote-tracking branch 'origin/master' into sp-2747

          GitLab Commit - SP-2747: move docker image to images and remove flake8

          GitLab Commit - SP-2747: Pinning OET version to 4.0.6

          GitLab Commit - SP-2747: push change in version

          GitLab Commit - SP-2747: re-enable teardown test

          GitLab Commit - SP-2747: refactored obsconfig

          GitLab Commit - SP-2747: remove auto use fixtures

          GitLab Commit - SP-2747: remove live logging

          GitLab Commit - SP-2747: remove redundant loop in asyncio

          GitLab Commit - SP-2747: remove sdp storage

          GitLab Commit - SP-2747: remove SKA from dish allocation

          GitLab Commit - SP-2747: remove skipped tests

          GitLab Commit - SP-2747: remove temp code

          GitLab Commit - SP-2747: remove temp settings

          GitLab Commit - SP-2747: remove typo

          GitLab Commit - SP-2747: remove xfail

          GitLab Commit - SP-2747: Resolved review comments

          GitLab Commit - SP-2747: revert temp changes

          GitLab Commit - SP-2747: Reverting EXIT_AT_FAIL

          GitLab Commit - SP-2747: Reverting marks

          GitLab Commit - SP-2747: run sdp tests only

          GitLab Commit - SP-2747: run tests with timeout set

          GitLab Commit - SP-2747: set to exit at first fail

          GitLab Commit - SP-2747: skip failing test

          GitLab Commit - SP-2747: skip oet resource assign testing

          GitLab Commit - SP-2747: skip release resources test

          GitLab Commit - SP-2747: temporarily re-enable logs

          GitLab Commit - SP-2747: test at 60 timeout

          GitLab Commit - SP-2747: test oet suspect case

          GitLab Commit - SP-2747: update csp entrypoint

          GitLab Commit - SP-2747: update csp ln entrypoint

          GitLab Commit - SP-2747: update dependencies

          GitLab Commit - SP-2747: update image ver

          GitLab Commit - SP-2747: update lintiing

          GitLab Commit - SP-2747: update linting

          GitLab Commit - SP-2747: update linting

          GitLab Commit - SP-2747: update names with new sdp one

          GitLab Commit - SP-2747: update poetry

          GitLab Commit - SP-2747: update rev nr

          GitLab Commit - SP-2747: update sdp chart

          GitLab Commit - SP-2747: update SDP to v0.12.0

          GitLab Commit - SP-2747: update tmc

          GitLab Commit - SP-2747: update types

          GitLab Commit - SP-2747: update version

          GitLab Commit - SP-2747: use config from observaytion

          GitLab Commit - SP-2747: use fix version

          GitLab Commit - SP-2747: use generic assign resources method

          GitLab Commit - SP-2747: use observation for csp

          GitLab Commit - SP-2747: use updated image

          GitLab Commit - SP-2747: uses dedicated parsing for csp LN

          GitLab Commit - SP-2747: uses skampi image

          GitLab Commit - SP-2747: uses sub id explicitly

          GitLab Commit - SP-2747 adding test case for configure command

          GitLab Commit - SP-2747 update the SDP 0.4 schema configuration using CDM library

          GitLab Commit - SP-2747 updated tmc chart

          GitLab Commit - SP-2747 Updated TMC Chart

          GitLab Merge request - SP-2747: new sdp names and work with py3.10

          GitLab Merge request - SP-2747: update SDP to v0.12.1

          GitLab Merge request - YAN-1091 Utilities, Persistent Volume and Demo Changes

          Wiki Page

          Wiki Page Loading...

          Structure

            Activity

              People

                m.ashdown Ashdown, Mark
                b.mort Mort, Ben
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete1919.0
                  Total1919.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel