Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4287

Automatic scheduling of SRC Workload task benchmark runs

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      The SRC Workloads Gitlab repository hosts workloads that are representative of what users will be running at SRC sites. These are a suitable starting point to use for compute benchmarking, now that run metrics are captured in our monitoring infrastructure. This feature captures preparation work for running some of the tasks on HPC sites, and an initial demonstration of a means of running these on a regular schedule, similar to the DM functional tests.

       

      Show
      The SRC Workloads Gitlab repository hosts workloads that are representative of what users will be running at SRC sites. These are a suitable starting point to use for compute benchmarking, now that run metrics are captured in our monitoring infrastructure. This feature captures preparation work for running some of the tasks on HPC sites, and an initial demonstration of a means of running these on a regular schedule, similar to the DM functional tests.  
    • Hide

      For at least the source finding and image convolution tasks (and potentially mosaicking and pulsar-search tasks):

      AC1: Task can be executed by the Singularity container runtime engine - merge into main branch in repo

      AC2: Integration test and longer version of task run available via make target

      AC3: Task is run on regular cadence (e.g. at least daily) 

      Show
      For at least the source finding and image convolution tasks (and potentially mosaicking and pulsar-search tasks): AC1: Task can be executed by the Singularity container runtime engine - merge into main branch in repo AC2: Integration test and longer version of task run available via make target AC3: Task is run on regular cadence (e.g. at least daily) 
    • 2
    • 2
    • 0
    • Team_MAGENTA
    • Sprint 5
    • PI24 - UNCOVERED

    • PI24-PB SRCNet0.x example-workflows-and-benchmarks tests-compilation

    Description

      This feature captures work required to enable execution of SRC Workloads tasks as regular benchmarks of compute performance within the SRCNet. Given the range of tasks in the SRC Workloads repo, this will focus on a subset of these (2-3 tasks initially). In addition to enabling these tasks to run on HPC resources via the Singularity container runtime engine, this feature captures work in creating integration test, and small versions of the selected tasks, and deploying these on a regular cadence on one or more machines, to generate a regular stream of runtime metric data for the dashboard to display. This could make use of something like e.g. https://github.com/mcuadros/ofelia.

      This will enable these tasks to be run regularly across sites of the SRCNetwork (including on HPC resources), as part of a regular compute benchmarking exercise. The integration test versions will also validate the ability of the various resources to run the tasks successfully, as well as providing additional quality control on future merge requests.

      Attachments

        Issue Links

          Structure

            Activity

              People

                r.bolton Bolton, Rosie
                A.Clarke Clarke, Alex
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (40.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do24.0
                  In Progress   25.0
                  Complete46.0
                  Total815.0

                  Dates

                    Created:
                    Updated:

                    Structure Helper Panel