Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-702

Refactor the Imaging/Calibration Pipeline to improve reusability in the SDP MVP

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Data Processing
    • Hide

      Taking some time for technical debt and focusing on reusability would increase the value of the developed code.

      This work might also help clarify some of the necessary interfaces between the pipeline components and how they are built into an workflow that can be executed by the SDP MVP

      Show
      Taking some time for technical debt and focusing on reusability would increase the value of the developed code. This work might also help clarify some of the necessary interfaces between the pipeline components and how they are built into an workflow that can be executed by the SDP MVP
      • Code on-boarded into GitLab with a description of improvements and integrated into the SDP MVP (sdp-prototype)
    • 3
    • 3
    • 2.667
    • Team_NZAPP
    • Sprint 5
    • Hide

      At present, the IGC pipeline is still being set up to operate under kubernetes, as there were some initial difficulties with setting this up. More information about these difficulties are noted on tickets NZAPP-156 and NZAPP-198

      Documentation for the IGC pipeline is also steadily underway, and will be a continous work in progress due to the size of the IGC pipeline code base. More information can be found on ticket NZAPP-181

      Feedback has been provided to the SIM team regarding the installation and use of the SDP Prototype in isolation, which is exclusive of integration due to the aforementioned kubernetes troubles. More information about this is noted on ticket NZAPP-157

      Stretch goals were not realised as the primary goals were a priority. However, these will be realised in the next increment (PI 6), assuming the feature containing these tasks is approved (see: SP-909)

      Show
      At present, the IGC pipeline is still being set up to operate under kubernetes, as there were some initial difficulties with setting this up. More information about these difficulties are noted on tickets NZAPP-156 and NZAPP-198 Documentation for the IGC pipeline is also steadily underway, and will be a continous work in progress due to the size of the IGC pipeline code base. More information can be found on ticket NZAPP-181 Feedback has been provided to the SIM team regarding the installation and use of the SDP Prototype in isolation, which is exclusive of integration due to the aforementioned kubernetes troubles. More information about this is noted on ticket NZAPP-157 Stretch goals were not realised as the primary goals were a priority. However, these will be realised in the next increment (PI 6), assuming the feature containing these tasks is approved (see: SP-909 )
    • 5.6
    • PI24 - UNCOVERED

    • Team_NZAPP goal_D1 goal_S2

    Description

      Following --SP-417-- and the SP-432 integration of the Imaging/Calibration Pipeline into the SDP MVP there are several features that would facilitate the reusability of the pipeline code.

      These include:

      • merge of pipeline code into single gitlab repo with automated testing and coverage,
      • better documentation for users,
      • finish the integration of the workflow into the SDP Evolutionary Prototype (Docker images + Helm chart + workflow script),
      • provide feedback to SIM on the draft SDP guide for incorporating workflows into the SDP MVP.

      Stretch:

      • moving the pipeline configuration parameters from a struct (requiring recompilation) to a separate configuration file more accessible to the SDP Evolutionary Prototype,
      • options to automatically pre-generate required assets such as the PSF (instead of it being generated manually by a separate execution of the pipeline),
      • allowing visibilities to be provided in formats (other than basic CSV files) and from other sources.

      Attachments

        Issue Links

          Structure

            Activity

              People

                b.mort Mort, Ben
                A.Ensor Ensor, Andrew [X] (Inactive)
                Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 3.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete920.0
                  Total920.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel