Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4384

Improve performance and scalability of calibration in ICAL

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Enabler
    • Should have
    • PI23
    • COM SDP SW
    • Data Processing
    • Hide

      Copied from Miro (see: https://miro.com/app/board/uXjVKZ900uo=/?moveToWidget=3458764589180836457&cot=14 ): We want our ICAL pipelines to be able to keep up with the Low and Mid telescopes at AA2 scale, i.e., these pipelines should take at most twice the duration of the processed observation. We are currently a significant factor away from that goal.

      • The most recent benchmarking results for the Low pipeline show that the workload for calibration is increasing each cycle due to the increasing complexity of calibration due to an improving sky model. This is confirmed by the timings from the Rapthor pipeline which also include a 4th and 5th cycle with the latter using 100% of the data instead of ~20% of the data. This means that calibration could well become the main bottleneck of the pipeline.
      • The most recent benchmarking results for the Mid pipeline show that the predict stage in WSClean does not scale to multiple node (it actually becomes slower). This needs to be investigated and addressed.
      Show
      Copied from Miro (see: https://miro.com/app/board/uXjVKZ900uo=/?moveToWidget=3458764589180836457&cot=14 ): We want our ICAL pipelines to be able to keep up with the Low and Mid telescopes at AA2 scale, i.e., these pipelines should take at most twice the duration of the processed observation. We are currently a significant factor away from that goal. The most recent benchmarking results for the Low pipeline show that the workload for calibration is increasing each cycle due to the increasing complexity of calibration due to an improving sky model. This is confirmed by the timings from the Rapthor pipeline which also include a 4th and 5th cycle with the latter using 100% of the data instead of ~20% of the data. This means that calibration could well become the main bottleneck of the pipeline. The most recent benchmarking results for the Mid pipeline show that the predict stage in WSClean does not scale to multiple node (it actually becomes slower). This needs to be investigated and addressed.
    • Hide
      1. Introduce image-based prediction in calibration by using the w-gridder (DUCC) in a DP3 Predict Step.
      2. The memory usage of DP3 is considered carefully, so that DP3 does not start swapping. A mechanism is thus introduced to balance gridding efficiency (favours large time and frequency intervals) and memory usage (favours short time and frequency intervals).
      3. The bad scaling of the predict stage in WSClean in the Mid pipeline is addressed.
      Show
      Introduce image-based prediction in calibration by using the w-gridder (DUCC) in a DP3 Predict Step. The memory usage of DP3 is considered carefully, so that DP3 does not start swapping. A mechanism is thus introduced to balance gridding efficiency (favours large time and frequency intervals) and memory usage (favours short time and frequency intervals). The bad scaling of the predict stage in WSClean in the Mid pipeline is addressed.
    • 6
    • 8
    • 0
    • Team_PANDO, Team_SCHAAP
    • Sprint 5
    • PI23 - UNCOVERED

    • AA2

    Attachments

      Structure

        Activity

          People

            p.wortmann Wortmann, Peter
            f.graser Graser, Ferdl
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Feature Progress

              Story Point Burn-up: (0%)

              Feature Estimate: 6.0

              IssuesStory Points
              To Do26.0
              In Progress   26.0
              Complete00.0
              Total412.0

              Dates

                Created:
                Updated:

                Structure Helper Panel