Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1053

Exploration of Baseline-Dependent Averaging (BDA) in pre-processing

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Data Processing
    • Hide

      BDA has the potential to significantly reduce visibility data volume and, as such, can help to overcome some of the I/O bottlenecks between processing stages.

      This ticket makes an important stride to integrating BDA as early in the processing chain as possible.

      Show
      BDA has the potential to significantly reduce visibility data volume and, as such, can help to overcome some of the I/O bottlenecks between processing stages. This ticket makes an important stride to integrating BDA as early in the processing chain as possible.
    • Hide
      • Definition of a logical data model to capture baseline-dependently averaged data (ideally applicable beyond MSv3)
      • Demonstration of BDA enabled DPPP, where DPPP reads a full resolution MS and stores BDA averaged MS, and vice versa (DPPP reads BDA data and stores full resolution MS).
      • Stretch: Measurement of efficiency gains (storage and processing)
      Show
      Definition of a logical data model to capture baseline-dependently averaged data (ideally applicable beyond MSv3) Demonstration of BDA enabled DPPP, where DPPP reads a full resolution MS and stores BDA averaged MS, and vice versa (DPPP reads BDA data and stores full resolution MS). Stretch: Measurement of efficiency gains (storage and processing)
    • 4
    • 4
    • 1.25
    • Team_SCHAAP
    • Sprint 5
    • Hide

      The code lives on https://git.astron.nl/RD/DP3/ and the BDA code has been merged to the development branch. Documentation is in https://git.astron.nl/RD/DP3/-/blob/development/docs/schemas/BDAAverager.yml . Because the development branch cannot yet be merged to master, the documentation is not deployed yet to https://www.astron.nl/citt/DP3

      Show
      The code lives on https://git.astron.nl/RD/DP3/ and the BDA code has been merged to the development branch. Documentation is in https://git.astron.nl/RD/DP3/-/blob/development/docs/schemas/BDAAverager.yml . Because the development branch cannot yet be merged to master, the documentation is not deployed yet to https://www.astron.nl/citt/DP3
    • 7.6
    • Stories Completed, Outcomes Reviewed, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO

    • PI22 - UNCOVERED

    • Team_SCHAAP
    • SPO-573 SPO-575

    Description

      Baseline-Dependent Averaging (BDA) can be used to reduce the visibility data volume significantly and can therefore be a useful tool to mitigate some of the I/O bottlenecks between the processing stages of the SDP pipelines.

      Key aspects to consider are:

      1. the definition of keywords in the MSv3 meta-data to describe the BDA scheme used, such that further processing stages do not have to read through the actual data to infer that crucial bit of information.
      2. integration of BDA with other pre-processing steps that are commonly done in early averaging of the raw visibility data such as RFI flagging and phase centering.

      These capabilities will be demonstrated by integrating BDA in a pre-processing pipeline.

      Attachments

        Issue Links

          relates to

          Enabler - An Enabler supports the activities needed to extend the Architectural Runway to provide future business functionality. These include exploration, architecture, infrastructure, and compliance. SP-1219 Develop initial implementation of BDA extension to MSv3 following BDA extension proposal

          • Could have - Items labelled as "Could have" are desirable but not necessary and could improve the user experience or customer satisfaction for a little development cost. These will typically be included if time and resources permit.
          • Funnel

          Enabler - An Enabler supports the activities needed to extend the Architectural Runway to provide future business functionality. These include exploration, architecture, infrastructure, and compliance. SP-601 VODML Toolchain for MSv3 API in C++ and Python

          • Could have - Items labelled as "Could have" are desirable but not necessary and could improve the user experience or customer satisfaction for a little development cost. These will typically be included if time and resources permit.
          • Discarded
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

          Structure

            Activity

              People

                p.wortmann Wortmann, Peter
                s.wijnholds Stefan Wijnholds
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 4.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete919.5
                  Total919.5

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel