Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4566

Improve Karabo usability and applicability for SRCNet Test Data

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      SRCNet and Construction side actitivities both have a common need for good quality, reproducible, documented simulated test data. While the details of the requirements on both sides will be slightly different, we can start to share these requirements so as to have overlap and commonality in the tooling we use for the same.

      At present, simulated test data generation is done in a slightly ad-hoc manner by astronomers, using different tools for the different stages of data generation and data manipulation sometimes required in between. This approach has an impact on reproducibility and provenance of the data generated.

      The Karabo pipeline has been demonstrated in PI23 (See here).

      This feature is to improving the usability, quality, and applicability of the data generated by the Karabo pipeline as informed by the stakeholders/demo feedback. 

      • Usability - reduce container image size
      • Quality - Add thermal noise to the data generated
      • Applicability - Add more AA layouts to the ones available (part of SP-4565 now)

      Sharing this work with folks in the DP ART, including the OSKAR and RASCIL developers will help with alignment with SDP/SRC alignment. We can then inform a Capability to align the stack of tools for simulated data generation. 

      Show
      SRCNet and Construction side actitivities both have a common need for good quality, reproducible, documented simulated test data. While the details of the requirements on both sides will be slightly different, we can start to share these requirements so as to have overlap and commonality in the tooling we use for the same. At present, simulated test data generation is done in a slightly ad-hoc manner by astronomers, using different tools for the different stages of data generation and data manipulation sometimes required in between. This approach has an impact on reproducibility and provenance of the data generated. The Karabo pipeline has been demonstrated in PI23 (See here ). This feature is to improving the usability, quality, and applicability of the data generated by the Karabo pipeline as informed by the stakeholders/demo feedback.  Usability - reduce container image size Quality - Add thermal noise to the data generated Applicability - Add more AA layouts to the ones available (part of SP-4565 now) Sharing this work with folks in the DP ART, including the OSKAR and RASCIL developers will help with alignment with SDP/SRC alignment. We can then inform a Capability to align the stack of tools for simulated data generation. 
    • Hide

      AC: System Demo to DP ART 

      AC: Meeting with OSKAR and RASCIL developers

      AC: Example notebooks and support provided for a test science user to verify the simulated test data generation pipeline with the ability to add thermal noise to the simulated visibilities and images using RASCIL and OSKAR backends.

       

      Show
      AC: System Demo to DP ART  AC: Meeting with OSKAR and RASCIL developers AC: Example notebooks and support provided for a test science user to verify the simulated test data generation pipeline with the ability to add thermal noise to the simulated visibilities and images using RASCIL and OSKAR backends.  
    • 1.5
    • 1.5
    • 0
    • Team_CHOCOLATE
    • Sprint 5
    • PI24 - UNCOVERED

    • PI24-PB SRCNet0.x Team_Chocolate

    Description

      https://github.com/i4Ds/Karabo-Pipeline 

      Interaction and feedback with OSKAR and RASCIL developers on DP ART to provide better alignment of APIs. 

      SDP alignment value - who also need to produce test data. Current approaches mean changing tool at every stage.

      Notes from meeting with Nichol

      • If we have to stack data in image plane for skao data scales, would this still work? Having noise in the simulated data helps identify this.

      Attachments

        Issue Links

          Structure

            Activity

              People

                Nichol.Cunningham Cunningham, Nichol
                r.joshi Joshi, Rohini
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 1.5

                  IssuesStory Points
                  To Do1016.0
                  In Progress   00.0
                  Complete00.0
                  Total1016.0

                  Dates

                    Created:
                    Updated:

                    Structure Helper Panel