Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3528

Extend Jupyterhub deployment via the Kubernetes ServiceMesh implementation

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      We have seen several instances of Jupyterhub deployments that allow a site to expose its resources (pods on nodes often in the form of VMs) to user workflows. This feature aims to extend a Jupyterhub deployment to multiple sites via the Kubernetes plane. This could be done in several ways but keeping the wider SRCNet context in mind, it is important that sites are given as much autonomy as possible over the way they expose resources. Implementing a ServiceMesh allows clusters to be deployed in a way that site prefers, including choice of CNI, connecting them, providing secure communication across services running in different clusters and visibility of the same.
      In the prototyping activities, we are yet to make an attempt at distributing any compute, and this would be the first step in that direction, to show an action initiated via one site that deploys some compute at a remote site. 

      Show
      We have seen several instances of Jupyterhub deployments that allow a site to expose its resources (pods on nodes often in the form of VMs) to user workflows. This feature aims to extend a Jupyterhub deployment to multiple sites via the Kubernetes plane. This could be done in several ways but keeping the wider SRCNet context in mind, it is important that sites are given as much autonomy as possible over the way they expose resources. Implementing a ServiceMesh allows clusters to be deployed in a way that site prefers, including choice of CNI, connecting them, providing secure communication across services running in different clusters and visibility of the same. In the prototyping activities, we are yet to make an attempt at distributing any compute, and this would be the first step in that direction, to show an action initiated via one site that deploys some compute at a remote site. 
    • Hide

      AC1: Kubernetes clusters are set up across different physical sites independently. A single Jupyterhub deployment with a hub service mirrored across the two (or more) sites.

      AC2: A single Jupyterhub deployment with user profiles that spin up user pods on different physical sites given some criteria.

      Show
      AC1: Kubernetes clusters are set up across different physical sites independently. A single Jupyterhub deployment with a hub service mirrored across the two (or more) sites. AC2: A single Jupyterhub deployment with user profiles that spin up user pods on different physical sites given some criteria.
    • Team_CORAL
    • Sprint 5
    • Show
      COR-389 [ServiceMesh] (UKSRC) K8s preparation COR-390 [ServiceMesh] (SPSRC) K8s preparation COR-390 [ServiceMesh] (SWSRC) K8s preparation COR-392 [ServiceMesh] (CHSRC) K8s preparation https://confluence.skatelescope.org/display/SRCSC/COR-446+%5BServiceMesh%5D+Configure+the+ServiceMesh+tool?src=contextnavpagetreemode -Alignment sessions: https://confluence.skatelescope.org/display/SRCSC/COR-451+%5BServiceMesh%5D+Alignment+sessions?src=contextnavpagetreemode
    • 21.4
    • Stories Completed, Outcomes Reviewed, Demonstrated, Accepted by FO
    • PI23 - UNCOVERED

    • PI19-PB PI20-PB SRC-CompPlat multi-team

    Description

      The aim is to run user workloads across different physical sites from one platform. This could be with Jupyterhub user profiles to start with to run user pods on different sites, before we build in the intelligence to pick a location based on workflow requirements.

      Anticipate Magenta team lead, with support from Coral (SPSRC?). Involved sites being in similar time zones will make the collaboration easier, large physical distances between the involved sites is not of particular importance at this stage since we are looking at functionality and feasibility, not performance. 

      This will entail 

      • physically distanced, independently deployed k8s planes accessible to each other (may be achieved by being externally accessible)
      • install linkerd with shared trust anchors on both clusters
        • check proxy injection
      • install multi cluster extension on both
      • enable exporting the hub service to create a service mirror, add traffic split object
        • this will show load balancing
      • Add few different user profiles that will map to pods being spun up at different physical sites

      Attachments

        Issue Links

          Structure

            Activity

              People

                r.joshi Joshi, Rohini
                r.joshi Joshi, Rohini
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 0.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete820.0
                  Total820.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel