Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-2993

Clarify AA2 scaling challenge datasets specifications

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Enabler
    • Should have
    • PI17
    • COM SDP SW
    • None
    • Data Processing
    • Hide

      Who? Developers working towards AA2 processing challenge

      What? A clear idea how datasets are going to get generated, and what properties we are going to be able to expect from them

      Why? To make the challenge meaningful, we need to ensure that we set the bar at the right height.

      Show
      Who? Developers working towards AA2 processing challenge What? A clear idea how datasets are going to get generated, and what properties we are going to be able to expect from them Why? To make the challenge meaningful, we need to ensure that we set the bar at the right height.
    • Hide
      • Evaluate existing dataset specifications for Low and Mid and revise to ensure we can suitably stress an AA2-scale imaging pipeline
        • Should be designed to pose non-trivial problems in calibration (direction-dependent), imaging and deconvolution.
        • Non-goal: Realism.
      • Provide solution for generating datasets that can be run by anyone on the DP testing platform.
      • Demonstrate by generating "decimated" data sets for development.
      Show
      Evaluate existing dataset specifications for Low and Mid and revise to ensure we can suitably stress an AA2-scale imaging pipeline Should be designed to pose non-trivial problems in calibration (direction-dependent), imaging and deconvolution. Non-goal: Realism. Provide solution for generating datasets that can be run by anyone on the DP testing platform. Demonstrate by generating "decimated" data sets for development.
    • Intra Program
    • 3
    • 3
    • 0
    • Team_YANDA
    • Sprint 5
    • Hide

      AC1: as discussed in the slide deck the assessment of data volumes, data rates and FLOPs showed agreement with the values given on the data challenge page; Data IO appears to be a missing dimension worth tracking though. Furthermore, this write-up also suggests how to stress a processing pipeline.

      AC2 + AC3: A demo and discussion at a Processing CoP meeting provided an explorative pratical study of various OSKAR paremeter settings by our new team member Nadia.

      Feedback by the FO led to an update of the k8s deployment setup (YAN-1200).

       

      Show
      AC1: as discussed in the slide deck the assessment of data volumes, data rates and FLOPs showed agreement with the values given on the data challenge page ; Data IO appears to be a missing dimension worth tracking though. Furthermore, this write-up also suggests how to stress a processing pipeline. AC2 + AC3: A demo and discussion at a Processing CoP meeting provided an explorative pratical study of various OSKAR paremeter settings by our new team member Nadia. Feedback by the FO led to an update of the k8s deployment setup ( YAN-1200 ).  
    • 18.2
    • Stories Completed, Integrated, Solution Intent Updated, BDD Testing Passes (no errors), Outcomes Reviewed, NFRS met, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO

    • PI24 - UNCOVERED

    Description

      The idea is to focus on simple dataset specifications initially, and do prep-work towards being able to generate full datasets at a later date.

      While the obvious approach here would be to use simulations, we are not too interested in realism here. The most important quality is that we gain the ability to easily and quickly generate datasets that put certain kinds of stress on our pipelines. This might as appropriate simplify our approach to - say - sky, beam or ionospheric models.

      Attachments

        Structure

          Activity

            People

              p.wortmann Wortmann, Peter
              D.Fenech Fenech, Danielle
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Feature Progress

                Story Point Burn-up: (100.00%)

                Feature Estimate: 3.0

                IssuesStory Points
                To Do00.0
                In Progress   00.0
                Complete714.5
                Total714.5

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Structure Helper Panel