Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3205

Transition from using Casacore arrays to XTensor in DPBuffer

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Data Processing
    • Hide

      The transition from the use of Casecore arrays to XTensor in DP3 will

      • improve maintainability of the code by avoiding Casacore reference semantics;
      • improve scalability by using the thread safe XTensor library;
      • improve readability of the code (and hence improve its maintainability and reduce development time);
      • enable use of various code optimisations from the XTensor library or libraries depending on XTensor for computational performance improvements;
      • Reduce the dependency of DP3 on Casacore.
      Show
      The transition from the use of Casecore arrays to XTensor in DP3 will improve maintainability of the code by avoiding Casacore reference semantics; improve scalability by using the thread safe XTensor library; improve readability of the code (and hence improve its maintainability and reduce development time); enable use of various code optimisations from the XTensor library or libraries depending on XTensor for computational performance improvements; Reduce the dependency of DP3 on Casacore.
    • Hide
      • The first three steps in the transition path described on https://confluence.skatelescope.org/display/SE/Transition+to+XTensor have been implemented in all DPSteps, so that XTensor can be used but DP3 still supports use of Casacore arrays as well (and thus remains backward compatible).
      • A follow-up feature has been defined to start work on steps 4 through 7 on the transition path.
      Show
      The first three steps in the transition path described on https://confluence.skatelescope.org/display/SE/Transition+to+XTensor  have been implemented in all DPSteps, so that XTensor can be used but DP3 still supports use of Casacore arrays as well (and thus remains backward compatible). A follow-up feature has been defined to start work on steps 4 through 7 on the transition path.
    • Intra Program
    • 5
    • 5
    • 0
    • Team_SCHAAP
    • Sprint 5
    • Hide

      Due to an unforeseen capacity reduction, not all DPSteps are using unique pointers and XTensor views yet. The linked stories give a nice overview of the work done. A few stories are still in progress and are likely going to be completed before PI planning. The remaining stories will roll over to PI19, for which a follow-up feature (SP-3413) was created. This follow-up ticket also includes the actual removal of Casacore arrays from the DPBuffer, which will then work with XTensors instead.

      Show
      Due to an unforeseen capacity reduction, not all DPSteps are using unique pointers and XTensor views yet. The linked stories give a nice overview of the work done. A few stories are still in progress and are likely going to be completed before PI planning. The remaining stories will roll over to PI19, for which a follow-up feature ( SP-3413 ) was created. This follow-up ticket also includes the actual removal of Casacore arrays from the DPBuffer, which will then work with XTensors instead.
    • 21.6
    • Stories Completed, Integrated, Outcomes Reviewed, Accepted by FO
    • PI24 - UNCOVERED

    • Team_SCHAAP
    • SDP-G2

    Description

      The DPBuffer object currently relies on Casacore arrays using reference semantics. The behaviour of these reference semantics is not always clear (affecting maintainability of the code) and these arrays are not thread safe (affecting scalability). These issues can be avoided by XTensor. The transition to XTensor has the additional advantage that XTensor has nice semantics (improving maintainability of the code as well as speed of code development). This enables the use of XTensor features, such as SIMD instructions, and the use of libraries for, e.g., GPU support exploiting XTensor (improving computational performance). Ultimately, this transition also reduces the dependency of DP3 on Casacore.

      A plan to make a smooth transition to XTensor is described on https://confluence.skatelescope.org/display/SE/Transition+to+XTensor. Team Schaap started to execute this plan in sprints 17.4 and 17.5. This can be continued in the following PIs. One of the key aspects of this plan is that the transition can be made in the course of several PIs, i.e., with limited effort per PI, because it does not block other work.

      Attachments

        Issue Links

          Structure

            Activity

              People

                D.Fenech Fenech, Danielle
                s.wijnholds Wijnholds, Stefan
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 5.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete1631.0
                  Total1631.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel