Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1178

Test updated visibility receive code with preallocated nodes

    XporterXMLWordPrintable

Details

    • Enabler
    • Won't have (this time)
    • None
    • COM SDP SW
    • None
    • Data Processing
    • Hide

      Use of a fixed pool of preallocated nodes for visibility receive is potentially a good solution

      1. allow for a fixed pool of IPs to be used, improving reaction speed in receive address exchange.
      2. to reduce overheads and delays in deploying a receive workflow and therefore make SDP more responsive when being requested to ingest data from CBF (CSP).

      Especially as SDP is going to be scheduling heavy processing and networking workloads, in which case we likely want to have specific (numbers of) processes to specific nodes anyway.

      Show
      Use of a fixed pool of preallocated nodes for visibility receive is potentially a good solution allow for a fixed pool of IPs to be used, improving reaction speed in receive address exchange. to reduce overheads and delays in deploying a receive workflow and therefore make SDP more responsive when being requested to ingest data from CBF (CSP). Especially as SDP is going to be scheduling heavy processing and networking workloads, in which case we likely want to have specific (numbers of) processes to specific nodes anyway.
    • Hide
      • Test configuration, deployment and operation of an SDP receive workflow using (multiple) preallocated nodes - generating usable receive addresses in the process.
      • Look into discovery / organisation and especially re-labelling of existing nodes. (This will eventually need to be hooked up into a resource allocation / assignment mechanism in the processing controller)
      • Compare results to previous work using fully dynamic node placement and making suggestions for future updates to this workflow as integrated into the SDP prototype code.
      • Stretch: Add support for determining receive addresses to workflow library (SP-1162), possibly via platform interface (Helm deployer)
      • Stretch: Coordinate with SP-1155 and SP-1157 to ingest visibilities from CBF receiver in a distributed fashion.
      Show
      Test configuration, deployment and operation of an SDP receive workflow using (multiple) preallocated nodes - generating usable receive addresses in the process. Look into discovery / organisation and especially re-labelling of existing nodes. (This will eventually need to be hooked up into a resource allocation / assignment mechanism in the processing controller) Compare results to previous work using fully dynamic node placement and making suggestions for future updates to this workflow as integrated into the SDP prototype code. Stretch: Add support for determining receive addresses to workflow library ( SP-1162 ), possibly via platform interface (Helm deployer) Stretch: Coordinate with SP-1155 and SP-1157 to ingest visibilities from CBF receiver in a distributed fashion.
    • 1
    • 8
    • 22.4

    • PI22 - UNCOVERED

    • SPO-726

    Attachments

      Issue Links

        Child Of

        Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SS-30 Platform & Networks

        • Not Assigned - Priority is not assigned yet.
        • Done
        depends on

        Enabler - An Enabler supports the activities needed to extend the Architectural Runway to provide future business functionality. These include exploration, architecture, infrastructure, and compliance. SP-1023 Proof-of-concept of visibility streaming over apache arrow/plasma for real-time processing

        • Not Assigned - Priority is not assigned yet.
        • Done
        relates to

        Enabler - An Enabler supports the activities needed to extend the Architectural Runway to provide future business functionality. These include exploration, architecture, infrastructure, and compliance. SP-1048 Assess performance latency needed to launch a containerised (receive) workflow using k8s

        • Not Assigned - Priority is not assigned yet.
        • Done

        Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-1155 Complete integration of visibility receive workflow

        • Not Assigned - Priority is not assigned yet.
        • Done

        Structure

          Activity

            People

              p.wortmann Wortmann, Peter
              b.mort Mort, Ben
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Feature Progress

                Story Point Burn-up: (0%)

                Feature Estimate: 1.0

                IssuesStory Points
                To Do00.0
                In Progress   00.0
                Complete00.0
                Total00.0

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Structure Helper Panel