UKSRC - Deploy all v0.1 local compulsory services

For this feature, these deployments may be hosted on temporary infrastructure and/or infrastructure that does not meet the full operational requirements committed to via the v0.1 EoIs

Some of these deployments may already be completed. The size of this feature will vary per SRC. Do not include optional services as part of this feature.

See the implementation plan doc and miro for additional information and ongoing SRC plans.

cavern has been conditionally added to the compulsory service list. Early PI24 development work / architectural decisions will answer if this is required to deliver the Data Management API and/or SODA Service.

Deployment is intended to be a simple process. Additional development work required, per service or per SRC, may be split out into other features or contained here - whichever approach best enables teams to plan their efforts.

For some local compulsory services, the technology stack is not fixed. If a technology is chosen that differs from proven implementations, it is the SRCs responsibility to plan and complete any additional work necessary to integrate their deployment with SRCNet's global services, including service monitoring. 

Multiple other features are required to enable this work and should be linked and sequenced accordingly, IBNLT SP-4598, SP-4570, SP-4570, SP-4569, SP-4517

Cross team details available in miro:
https://miro.com/app/board/uXjVK0qVHVs=/ 

The DAAC team have been identified as the team to help operate a number of the services required for SRCNet 0.1 at RAL:

• Implement GitOps for managing staging and production K8s clusters
• SRCNet Azimuth GitOps managed across both RAL and Cambridge from a single git repo.
• Deployment of Data Management API, SODA, JuypterHub, Visualization service, and any associated monitoring.

Do not do this former stretch AC as part of this feature:

• Hopefully setup SRCNet Gateway, CANFAR with Cavern

IF you think the team has the capacity to deploy optional services, propose this as a separate, Could Have feature.

The Purple team (via help from other RAL teams) will be responsible for:

• (Running central SKA IAM service)
• Compute hardware enrolled into the STFC RAL OpenStack based Cloud
• Storage hardware turned into a Ceph cluster (assuming we start with Manila+xrootd, what is team DAAC given)
• FTS and Rucio RSE created on top of the above storage hardware
• RAL cloud, should authenticate via SKA IAM, with group membership within SKA IAM authorizing access to OpenStack projects
• Providing Team DAAC with appropriate access to run the other SRCNet services, particular those requiring access to the Rucio RSE.

The DAAC team need to bring up all the other services at RAL, building on the above storage, including:

• Bootstrap GitOps (flux) managed K8s to manage helm deployments:
  https://github.com/stackhpc/capi-helm-fluxcd-config 
• Setup second cluster for production, and test promoting changes from staging into production
• Using these docs: https://srcnet-software-docs.readthedocs.io/en/latest/ 
  • Visualisation
  • Jupyter Hub (linked to Rucio RSE)
  • Monitoring
  • Science Gateway (with Chris Skipper's help)
  • CANFAR, CAVERN?
  • CVMFS Client
  • Azimuth