Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3164

Resolve SDP configuration library etcd client issues

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Enabler
    • Must have
    • PI18
    • COM SDP SW
    • None
    • True
    • Data Processing
    • Hide

      See Why? in description

      Show
      See Why? in description
    • Hide

      See What? in description

      Show
      See What? in description
    • 3
    • 3
    • 0
    • Team_ORCA
    • Sprint 5
    • Hide

      Presented at System Demo 18.6 (slides)

      The feature has been down-scoped to the implementation of primitive methods on the backend code as well as the full implementation of the Etcd3Transaction class. Work on the Watcher and final integration is moved to next PI.

      AC1:

       

      AC2:

      • Not achieved; because we have only implemented part of the new backend, we have not yet integrated the updates with SDP and the processing scripts.
      Show
      Presented at System Demo 18.6 ( slides ) The feature has been down-scoped to the implementation of primitive methods on the backend code as well as the full implementation of the Etcd3Transaction class. Work on the Watcher and final integration is moved to next PI. AC1: Spikes carried out to familiarize the team with the following (documentation on Confluence): gRPC: https://confluence.skatelescope.org/display/SE/Investigation+of+gRPC python-etcd3: https://confluence.skatelescope.org/display/SE/Investigation+of+python-etcd3+client+package existing Configuration Library code: https://confluence.skatelescope.org/display/SE/SDP+Configuration+Library A set of abstract classes have been created to formalize the backend API ([MR52| https://gitlab.com/ska-telescope/sdp/ska-sdp-config/-/merge_requests/52 ]) Implementation of primitives using the python-etcd3 client package: MR53 , MR59 , MR61 , MR62 , MR63 Implementation of the Etcd3Transaction class: MR58 , MR59 , MR61 , MR63 At the end of the PI, all original tests were ran with the new code bugs have been fixed: MR65 Additional work related to revisions and leases: MR56 and https://confluence.skatelescope.org/display/SE/Investigation+of+%27alive%27+method A performance test was developed to prove that the new client packages brings improvements to SDP performance as expected, even with the current partial implementation. Results are documented on Confluence: https://confluence.skatelescope.org/display/SE/Performance+Test+on+new+etcd+backend   AC2: Not achieved; because we have only implemented part of the new backend, we have not yet integrated the updates with SDP and the processing scripts.
    • 22.3
    • Stories Completed, Integrated, Solution Intent Updated, BDD Testing Passes (no errors), Outcomes Reviewed, NFRS met, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO
    • PI24 - UNCOVERED

    • SOL-G4

    Description

      The etcd3-py client package that is used by the configuration library is no longer maintained; the last release was in April 2019. We have found problems with its performance that cause the controller components, especially the processing controller, to slow down significantly as more entries are added to the configuration DB (see SKB-180). For these reasons we need to replace the client package with one that is better maintained.

      A promising candidate for this is python-etcd3. This client uses gRPC to connect to the server, so if we find the high-level interface is not suitable for doing certain things we could bypass it by using gRPC directly.

      A stretch acceptance criterion is to optimise the performance of the watcher by using the updates sent by the server to cache values on the client side. This depends on fixing a bug in the server to do with the ordering of the watch responses and progress notifications that it sends.

      Who?

      • SDP developers and users.

      What?

      • New etcd backend for configuration library using alternative client package.
      • Releases of the SDP and its processing scripts using the new backend.
      • Stretch: optimise watcher by caching values from watch responses on the client side.

      Why?

      • Etcd client package (etcd3-py) used by configuration library is no longer maintained and has significant performance issues (see SKB-180).

      Attachments

        Issue Links

          Structure

            Activity

              People

                p.wortmann Wortmann, Peter
                m.ashdown Ashdown, Mark
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 3.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete2248.0
                  Total2248.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel