Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-932

Measure CSP.LMC performance, investigate causes of performance degradation CLONE of SP-636

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Feature
    • Not Assigned
    • None
    • None
    • Obs Mgt & Controls
    • Hide

      Detect source of performance issues so that they can be addressed in early stages of development, so that if the issues cannot be fixed in current design, the SKA project can pivot to more suitable design choices. 

      Show
      Detect source of performance issues so that they can be addressed in early stages of development, so that if the issues cannot be fixed in current design, the SKA project can pivot to more suitable design choices. 
    • Hide

      **Get a metric of to what extent we can scale on the integration k8s cluster, using possibly simple devices that replicate the hierarchy with several hundred Dishes

      Synchronous commands replaced with asynchronous commands where required (see feature description). 

      Source of the performance issues identified. 

      Plan for mitigation / resolution proposed. 

      Show
      **Get a metric of to what extent we can scale on the integration k8s cluster, using possibly simple devices that replicate the hierarchy with several hundred Dishes Synchronous commands replaced with asynchronous commands where required (see feature description).  Source of the performance issues identified.  Plan for mitigation / resolution proposed. 
    • 2
    • 7.5
    • Sprint 5
    • 16.3
    • PI22 - UNCOVERED

    Description

      This is a CLONE of SP-636 intended to capture all the information and acceptance criteria since SP-636 has been reduced to a limited exploratory Spike based on available bandwidth. This feature can be taken up as a follow up after SP-636 is completed

       

      We consider a short timeboxed Spike of a Sprint to look at the measurement possibilities and As Is performance. Further work could be covered later (A cloned Feature SP-712 created to capture all the initial descriptions)

      Serious performance issues have been detected when a realistic number of CSP sub-arrays and other TANGO Servers and Devices are instantiated. The cause has not been identified, it may be inherent to pyTango implementation, caused by sub-optimal container configuration, blocking due to extensive use of forwarded attributes, use of synchronous (as opposed of asynchronous commands) commands. 

      The idea of this investigation is to measure the scalability and the point at which the performance degrades when several hundreds of Dishes with CBF and related Tango devices are added to the MVP.

      It would be useful to be able to automate the creation and testing of the entire hierarchy with a variable no of devices so that the performance test can be done on various scales and the point of bottleneck reached

      These are some of the options that will be explored to fix issues detected: 

      • Where appropriate replace synchronous with asynchronous commands (in particular when command completion depends on large number of other devices or involves parsing of large JSON  objects). 
      • Experiment using CSP.LMC and Mid.CBF images that instantiate large number of TANGO Servers/Devices.
      • Experiment with different container configurations.

       

      Attachments

        Issue Links

          Structure

            Activity

              People

                v.mohile Mohile, Vivek
                s.vrcic Vrcic, Sonja
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete00.0
                  Total00.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel