Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4256

Develop rucio-task-manager to ingest data using the ska-src-ingestion tool as part of tests

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      These tests will involve several components of the SRCNet and more than one SRCSite thus testing integration across these components and sites. They will be run multiple times, ideally regularly.

      Show
      These tests will involve several components of the SRCNet and more than one SRCSite thus testing integration across these components and sites. They will be run multiple times, ideally regularly.
    • Hide

      AC1: Updated tests saved to Rucio task manager repository, with explanation for execution.
      AC2: Dashboard views showing data ingestion, replication and the corresponding data transfer performance, confirming that the test works as intended.

      Show
      AC1: Updated tests saved to Rucio task manager repository, with explanation for execution. AC2: Dashboard views showing data ingestion, replication and the corresponding data transfer performance, confirming that the test works as intended.
    • 1
    • 1
    • 0
    • Team_MAGENTA
    • Sprint 3
    • Hide

      AC1: Code added to repo: https://gitlab.com/ska-telescope/src/ska-rucio-task-manager/-/blob/main/src/tasks/tests/ingestion.py

      Instructions for running tests are all given at top of repository: https://gitlab.com/ska-telescope/src/ska-rucio-task-manager 

      AC2: Dashboard added: https://monit.srcdev.skao.int/grafana/d/fdpc7obnbdkhsf/ingestion

      Demo: https://confluence.skatelescope.org/pages/viewpage.action?pageId=280401927 

       

      Show
      AC1: Code added to repo: https://gitlab.com/ska-telescope/src/ska-rucio-task-manager/-/blob/main/src/tasks/tests/ingestion.py Instructions for running tests are all given at top of repository: https://gitlab.com/ska-telescope/src/ska-rucio-task-manager   AC2: Dashboard added: https://monit.srcdev.skao.int/grafana/d/fdpc7obnbdkhsf/ingestion Demo: https://confluence.skatelescope.org/pages/viewpage.action?pageId=280401927    
    • 23.6
    • Stories Completed, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO

    • PI24 - UNCOVERED

    • SRC23-PB SRCNet0.1 data-ingestion-dissemination-and-replication data-lifecycle network-tests tests-compilation

    Description

      Test data (precursor data/synthetic test data) is ingested using the ingestion service, and replicated to sites using a set of sites. The task manager can be used for this purpose https://gitlab.com/ska-telescope/src/ska-rucio-task-manager 

      Attachments

        Issue Links

          Child Of

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SP-4871 Data Management v0.1 - Roadmap

          • Not Assigned - Priority is not assigned yet.
          • Implementing
          relates to

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-4291 Upload intermediate/derived data products back into Rucio for convolution task

          • Not Assigned - Priority is not assigned yet.
          • Funnel

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-4227 Define requirements for SRCNet Ingestion Nodes

          • Must have - Items labelled as "Must have" are critical to the current delivery timebox in order for it to be a success.
          • Done

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-4274 Develop metadata-based replication tests as part of regular functional testing

          • Must have - Items labelled as "Must have" are critical to the current delivery timebox in order for it to be a success.
          • Done
          mentioned in

          Page Loading...

          mentioned on

          GitLab Commit - Merge branch 'sp4256-fix-bugs' into 'main'

          GitLab Commit - Merge branch 'sp4256-ingestion-test-fix-metadata-validation' into 'main'

          GitLab Merge request - SP-4256 Fix bug in metadata validation, improve logging

          GitLab Merge request - SP-4256 Fix bug in metadata validation, improve logging

          GitLab Merge request - SP-4256 fix bugs

          Structure

            Activity

              People

                r.bolton Bolton, Rosie
                r.joshi Joshi, Rohini
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 1.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete610.0
                  Total610.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel