Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-431

Simple SDP Processing Controller prototype

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Data Processing
    • Hide

      The SDP Execution Control Processing Controller is central to providing the functionality required for orchestration of realtime and batch workflows. While initial groundwork for this has been started in PI#3, developing a functional prototype of this component is valuable for enabling future functionality in the SDP within the evolutionary prototype MVP.

      Show
      The SDP Execution Control Processing Controller is central to providing the functionality required for orchestration of realtime and batch workflows. While initial groundwork for this has been started in PI#3, developing a functional prototype of this component is valuable for enabling future functionality in the SDP within the evolutionary prototype MVP.
    • 2
    • 2
    • 2
    • Team_SIM
    • Sprint 4
    • 4.5
    • PI24 - UNCOVERED

    • DP_epic_1 Team_SIM goal4

    Description

      Be able to spawn containerised workflows (scripts) programmatically using the k8s helm API.

      By taking this approach it should be possible to demonstrate that a workflow script can be executed by the Processing Controller in the exact same way as using the helm CLI.

      Once an initial version of the Processing Controller has been developed and tested both independently and as part of the integrated MVP deployment, addition functions such as monitoring and reasoning about resources and queuing of batch workflows could be added to this component. This should first involve a discussion with the architecture team and/or SDP architect(s). Note that also for these considerations there is potentially an overlap with the workflow library and how workflows schedule resources for individual workflow stages.

      Proposed steps (not all need to be completed in PI#4)

      1. Split processing controller from Tango Subarray device
      2. Dynamic deployment of workflow scripts (Dealing with versions of workflow scripts)
      3. Move to PI-5: Implement (with real-time workflows) resource negotiation between workflow, processing controller backed by the platform interface API.
      4. Move to PI-5: Scheduling batch workflow scripts for execution

      Attachments

        Issue Links

          Structure

            Activity

              People

                p.wortmann Wortmann, Peter
                b.mort Mort, Ben
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete48.0
                  Total48.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel