Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4227

Define requirements for SRCNet Ingestion Nodes

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      BH1: Ingestion of data into the SRCNet is crucial for the testing of the SRCNet v0.1

      BH2: Multiple ingestion nodes will allow the ingestion of different types of data so analysis and provision of local resources requested are very relevant

      Show
      BH1: Ingestion of data into the SRCNet is crucial for the testing of the SRCNet v0.1 BH2: Multiple ingestion nodes will allow the ingestion of different types of data so analysis and provision of local resources requested are very relevant
    • Hide

      AC1: List of candidate ingestion endpoints

      AC2: Template of requirements per ingestion node as per described

      AC3: Discussion with ingestion nodes candidates contact points on the requirements requested for common understanding

      Show
      AC1: List of candidate ingestion endpoints AC2: Template of requirements per ingestion node as per described AC3: Discussion with ingestion nodes candidates contact points on the requirements requested for common understanding
    • 0.5
    • 0.5
    • 0
    • Team_MAGENTA
    • Sprint 3
    • Hide

      A document has been drawn up here: https://confluence.skatelescope.org/display/SRCSC/SP4227%3A+Requirements+for+SRCNet+Data+Ingestion+Nodes though it is likely this will continue to be developed.

      The only ingestion candidate identified so far is the site currently used by the Magenta team at STFC Cloud ('SKAOSRC')

      Show
      A document has been drawn up here: https://confluence.skatelescope.org/display/SRCSC/SP4227%3A+Requirements+for+SRCNet+Data+Ingestion+Nodes though it is likely this will continue to be developed. The only ingestion candidate identified so far is the site currently used by the Magenta team at STFC Cloud ('SKAOSRC')
    • 23.6
    • Stories Completed, Integrated, Outcomes Reviewed, Satisfies Acceptance Criteria, Accepted by FO

    • PI24 - UNCOVERED

    • SRC23-PB SRCNet0.1 data-lifecycle

    Description

      For SRCNet v0.1, there is a plan to have a set of ingestion nodes to allow the dissemination of data and registration of data into the SRCNet data lake

      This should cover all the candidates, at least:

      • Description of data to be included from this node, including size and, if applicable, production rate
      • Requirements on storage (staging area) and compute (ingestion daemon service)
      • Requirements on network connectivity with other nodes
      • Description of the metadata data model that could be provided for the data (e.g. IVOA Obscore metadata)

      Candidates will be locations close to the telescope locations (under the coordination of pipelines operators/team) and SRC nodes that will provide ingestion of data sources like synthetically produced data, precursors data, or other types

      In all the nodes, the software to be used will be the SRCNet ingestion software (and/or evolution):

      https://gitlab.com/ska-telescope/src/ska-src-ingestion

      This information will be provided to the candidates of ingestion nodes for analysis and provision.

       

       

       

      Attachments

        Issue Links

          Child Of

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SP-4871 Data Management v0.1 - Roadmap

          • Not Assigned - Priority is not assigned yet.
          • Implementing
          relates to

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-4256 Develop rucio-task-manager to ingest data using the ska-src-ingestion tool as part of tests

          • Must have - Items labelled as "Must have" are critical to the current delivery timebox in order for it to be a success.
          • Done

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-3680 Build and deploy a data ingestion service to complement the current data management API

          • Must have - Items labelled as "Must have" are critical to the current delivery timebox in order for it to be a success.
          • Done
          mentioned in

          Page Loading...

          Structure

            Activity

              People

                Jesus.Salgado Salgado, Jesus
                Jesus.Salgado Salgado, Jesus
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 0.5

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete34.0
                  Total34.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel