Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3098

CAR Caching proxies for upstream dependent artefacts for apt packages

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Services
    • Hide

      In order to have a reliable, secured supply chain any artefact required by SKAO should be hosted within SKAO where we can access it safely, quickly and maintain its lifecycle. To achieve this we need to host both our own artefacts and any upstream artefacts we depend on. This is also in line with 7 R's of a supply chain (in the logistics world: Getting the Right product, in the Right quantity, in the Right condition, at the Right place, at the Right time, to the Right customer, at the Right price).

       

      To achieve a complete supply chain, it's important we first start with CAR as a first step and trickle down to each location as proxies and caches.

      Show
      In order to have a reliable, secured supply chain any artefact required by SKAO should be hosted within SKAO where we can access it safely, quickly and maintain its lifecycle. To achieve this we need to host both our own artefacts and any upstream artefacts we depend on. This is also in line with 7 R's of a supply chain (in the logistics world: Getting the Right product, in the Right quantity, in the Right condition, at the Right place, at the Right time, to the Right customer, at the Right price) .   To achieve a complete supply chain, it's important we first start with CAR as a first step and trickle down to each location as proxies and caches.
    • Hide
      • Following types of upstream artefacts are cached in CAR by setting up proxies: apt
        • Seperate Debian Proxy packages into separate Nexus instances (behind apt.artefact.skao.int or something similar)
      • Binary packages used in ansible-collections are investigated and changed with in the order of availability: OCI images, apt packages, binary raw artefacts
      • CAR and proxies is connected with the appliances. i.e. 
        • playbooks are updated to use CAR as a proxy 
        • Servers are redeployed with the above config
      Show
      Following types of upstream artefacts are cached in CAR by setting up proxies: apt Seperate Debian Proxy packages into separate Nexus instances (behind apt.artefact.skao.int or something similar) Binary packages used in ansible-collections are investigated and changed with in the order of availability: OCI images, apt packages, binary raw artefacts CAR and proxies is connected with the appliances. i.e.  playbooks are updated to use CAR as a proxy  Servers are redeployed with the above config
    • 2
    • 2
    • 0
    • Team_IT, Team_SYSTEM
    • Sprint 5
    • Hide

      A new role is added for apt package configuration: https://gitlab.com/ska-telescope/sdi/ska-ser-ansible-collections/-/merge_requests/62

       

      This has been tested manually one of the k8s workers and update rolled out the rest of the machines, we haven't updated ubuntu20.04 machines as they will be replaced anyway and playbooks don't support any other version

      Show
      A new role is added for apt package configuration: https://gitlab.com/ska-telescope/sdi/ska-ser-ansible-collections/-/merge_requests/62   This has been tested manually one of the k8s workers and update rolled out the rest of the machines, we haven't updated ubuntu20.04 machines as they will be replaced anyway and playbooks don't support any other version
    • 17.6
    • Satisfies Acceptance Criteria, Accepted by FO

    • PI24 - UNCOVERED

    • Team_IT Team_SYSTEM

    Description

      The SKAO provides a Central Artefact Repository based on Nexus Repository Manager3.  This has built in capabilities for caching from upstream sources for most artefact types (formats) that it can support.  The key SKAO published artefacts are OCI Images, Python libraries, Conan packages, Helm charts, and Raw artefact types. 

      The SKAO will also provide mirroring of upstream official artefact repositories - these include OCI Images (docker.io, quay.io, gcr.io), Apt (Ubuntu), Helm (public, Ceph, GitHub, GitLab), PyPi.

      These two caches will provide an authoritative source and logistics support for SKAO related artefacts.  The PoP Nexus instance points to these to provide local caching services for any software deployments and will be integrated with deployment and software building processes in the PoP location - eg: Kubernetes, containers, Docker, Podman etc.

      The caching facility will help with the speed of deployment issues in geographically dispersed locations, and the cache can be primed to support use cases where it is necessary to avoid the first-hit latency problem.

      Attachments

        Issue Links

          is cloned by

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-3100 CAR Caching proxies for upstream dependent artefacts for helm charts

          • Low - Minor problem or easily worked around.
          • Analyzing
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

          Structure

            Activity

              People

                m.deegan Deegan, Miles
                U.Yilmaz Yilmaz, Ugur
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete713.0
                  Total713.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel