Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4188

Enable testing of high availability data transfer nodes for SRC Net 0.1

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • National SRC
    • Hide

      BH: A highly resilient and performant DTN Endpoint will allow for production-level performance to be demonstrated, and for further optimisation and prototyping work to be developed against.

      For SRCNet - "provides an endpoint that could be tested in anger", provides stable endpoints for receiving data.

      Show
      BH: A highly resilient and performant DTN Endpoint will allow for production-level performance to be demonstrated, and for further optimisation and prototyping work to be developed against. For SRCNet - "provides an endpoint that could be tested in anger", provides stable endpoints for receiving data.
    • Hide

      Must haves for UK:

      AC1: At least two DTN hosts are installed and provisioned

      AC2: Functional transfers between other demonstrator endpoints have been performed, using appropriate token-based authorisation and orchestrated data movement. In particular in connection with the Rucio testbed.

      Must have for SRCNet... but not urgent for this PI:

      Stretch: Documentation and recipies for other sites - is made available

       

      (Stretch, not needed for 0.1): A High Availability (clustered) setup is configured and demonstrated

       

       

      Show
      Must haves for UK: AC1: At least two DTN hosts are installed and provisioned AC2: Functional transfers between other demonstrator endpoints have been performed, using appropriate token-based authorisation and orchestrated data movement. In particular in connection with the Rucio testbed. Must have for SRCNet... but not urgent for this PI: Stretch: Documentation and recipies for other sites - is made available   (Stretch, not needed for 0.1): A High Availability (clustered) setup is configured and demonstrated    
    • 1
    • 1
    • 0
    • Team_PURPLE
    • Sprint 5
    • Hide

      Moving to releasing, due to the completion of the two AC.

      AC1: One XRootD server is installed and provisioned (and participating in functional tests within Rucio). 
      The additional XRootD servers are physically installed, but awaiting final network routing configuration and installation of software. 
      The single XRootD instance has already demonstrated ~ 50Gb/s throughput, and ~ 10Gb/s transfers from another SRCNet Node site. 

      AC2: Demonstration of successful transfer to the XRootD can be seen at: https://monit.srcdev.skao.int/grafana/d/BfW0IQQVk/rucio-events?orgId=1&var-src=All&var-dst=UKSRC_RAL_XRD_DEVCEPHFS&var-protocol=All&var-scopes=testing&var-filename=%2A&var-bin=$__auto_interval_bin&var-throughput_min_filesize_bytes=1000000&from=now-30d&to=now

       

      Initially identified stretch activities will be picked up next PI.

      Show
      Moving to releasing, due to the completion of the two AC. AC1: One XRootD server is installed and provisioned (and participating in functional tests within Rucio).  The additional XRootD servers are physically installed, but awaiting final network routing configuration and installation of software.  The single XRootD instance has already demonstrated ~ 50Gb/s throughput, and ~ 10Gb/s transfers from another SRCNet Node site.  AC2: Demonstration of successful transfer to the XRootD can be seen at: https://monit.srcdev.skao.int/grafana/d/BfW0IQQVk/rucio-events?orgId=1&var-src=All&var-dst=UKSRC_RAL_XRD_DEVCEPHFS&var-protocol=All&var-scopes=testing&var-filename=%2A&var-bin=$__auto_interval_bin&var-throughput_min_filesize_bytes=1000000&from=now-30d&to=now   Initially identified stretch activities will be picked up next PI.
    • PI23 - UNCOVERED

    • SRCNet0.1 operations-monitoring-and-network tests-compilation

    Description

      Bulk data transport forms the heart of SRCNet. Participating sites will need to deploy highly available resilient Data Transport Nodes (DTNs, and aka Gateways) to provide high-throughput data transfer from the Observatories to the SRCNodes, and between SRCNodes and individual sites. 

      This enabler will provide the ability to provide a HA DTN config for SRCNet test, development and (in principle) v0.1 capabilities at RAL. As well as provisioning for a production environment, a test host will also be provisioned for additional development activities. 

      Attachments

        Issue Links

          Structure

            Activity

              People

                J.Walder Walder, James
                J.Walder Walder, James
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 1.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete00.0
                  Total00.0

                  Dates

                    Created:
                    Updated:

                    Structure Helper Panel