Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4622

Consolidate iterative self-calibration (ICAL) pipeline

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Feature
    • High
    • PI24
    • COM SDP SW
    • Data Processing
    • Hide

      As a pipeline developer, I want to have a well-structured and maintainable code base.

      As (commissioning scientist), I want to be able to exercise the ICAL pipeline with minimal effort to install it and learn how it works.

      Show
      As a pipeline developer, I want to have a well-structured and maintainable code base. As (commissioning scientist), I want to be able to exercise the ICAL pipeline with minimal effort to install it and learn how it works.
    • Hide
      1. The work to add a reordering step to DP3 is completed.
      2. Supporting Python data models are added to ska-sdp-datamodels and functions are available through ska-sdp-func-python.
      3. Test coverage of the workflow repository is improved to 75% (currently 65%)
      4. The workflow repo is renamed to ska-sdp-wflow-selfcal
      5. The workflow and its configuration is documented in a manner that makes it usable / accessible to end users.
      6. A release of the pipeline is available to end users.
      7. Stretch: A spike is held to discuss the possibility if introducing w-towers into DP3 and/or WSClean
      Show
      The work to add a reordering step to DP3 is completed. Supporting Python data models are added to ska-sdp-datamodels and functions are available through ska-sdp-func-python. Test coverage of the workflow repository is improved to 75% (currently 65%) The workflow repo is renamed to ska-sdp-wflow-selfcal The workflow and its configuration is documented in a manner that makes it usable / accessible to end users. A release of the pipeline is available to end users. Stretch: A spike is held to discuss the possibility if introducing w-towers into DP3 and/or WSClean
    • Intra Program
    • 3
    • 3
    • 0

    • Component Version REL-1735 Initial release of SDP self-calibration pipeline

    • Team_DHRUVA, Team_PANDO, Team_SCHAAP
    • Sprint 5

    • PI24 - UNCOVERED

    • AA2 DP_ART_24.3

    Description

      Up to PI22, the Low and Mid iterative self-calibration (ICAL) pipelines were developed independently. In PI23, they were combined in a single ICAL pipeline that can process both Low and Mid data by providing another .yml configuration file. Over the PIs, there has been a big push to get to this point with a feature-complete ICAL pipeline and to make performance improvements. This forward drive, however, has resulted in many loose ends, including:

      • Adding a visibility reordering step to DP3 and connecting that to WSClean is not completed yet;
      • Supporting Python functions are still in the repo itself instead of in the PFL;
      • Test coverage is lower than desired / not adhering to SKA DoD;
      • The workflow repo should be renamed from ska-sdp-wflow-low-selfcal to ska-sdp-wflow-ical or similar;
      • End user documentation should be added and a release of the pipeline should be made.

       

       

      See frame on DP ART board: https://miro.com/app/board/uXjVK6Lrdw4=/?moveToWidget=3458764597569097998&cot=14 

      Attachments

        Issue Links

          Is delivered by

          Component Version - Component/module/library releases REL-1735 Initial release of SDP self-calibration pipeline

          • Planned

          Structure

            Activity

              People

                D.Fenech Fenech, Danielle
                f.graser Graser, Ferdl
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 3.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete00.0
                  Total00.0

                  Dates

                    Created:
                    Updated:

                    Structure Helper Panel