Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1176

Demonstrate RCAL and/or flagging using in-memory visibility streaming

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Enabler
    • Won't have (this time)
    • None
    • COM SDP SW
    • None
    • Data Processing
    • Hide

      Assess suitably of visibility streaming solution developed for SP-1023 to more real-life processing scenarios

      Show
      Assess suitably of visibility streaming solution developed for SP-1023 to more real-life processing scenarios
    • Hide
      • Processing component that accepts visibility stream and metadata via Apache Plasma (typically utilising the result of SP-841)
      • Stretch: Consider how to stream results back in a suitable way - obvious choice is to just write back into Plasma, but realistically we will want to work towards a solution that works non-locally here (i.e. "Data Queues" in the SDP architecture).
      • Stretch: Consider how to distribute computation. Especially relevant for real-time processing, where we might need to implement a distributed consensus algorithm.
      Show
      Processing component that accepts visibility stream and metadata via Apache Plasma (typically utilising the result of SP-841 ) Stretch: Consider how to stream results back in a suitable way - obvious choice is to just write back into Plasma, but realistically we will want to work towards a solution that works non-locally here (i.e. "Data Queues" in the SDP architecture). Stretch: Consider how to distribute computation. Especially relevant for real-time processing, where we might need to implement a distributed consensus algorithm.
    • 5
    • 5.6
    • 11.2

    • PI24 - UNCOVERED

    Description

      SP-1023 has demonstrated that we can stream data within processes involved in ingest/receive, and we likely do not have much more to prove for PSI or early AA deployments. However, eventually we clearly want to be able to use in-memory data exchange to introduce non-trivial processing into the workflow.

      For realtime pipelines, the obvious options are:

      • a form of real-time calibration component (to work towards supporting tied-array beamforming, critical to test early due to the many involved telescope sub-systems)
      • a flagging component (to explore more SDP-specific questions, especially around how we want to handle flagging data)

      Note that both SP-1174 and SP-841 must likely happen before this can realistically be started - if we attempt this we will likely want to de-scope.

      Attachments

        Issue Links

          relates to

          Spike - A time limited investigation to get some knowledge SP-461 Support ADR to determine Initial Calibration Feedback to CBF

          • Must have - Items labelled as "Must have" are critical to the current delivery timebox in order for it to be a success.
          • Discarded
          mentioned in

          Page Loading...

          Page Loading...

          Structure

            Activity

              People

                p.wortmann Wortmann, Peter
                b.mort Mort, Ben
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 5.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete00.0
                  Total00.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel