Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-4167

SRC workloads Gitlab repo technical debt

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      The SRC workloads Gitlab repository needs to accommodate various ways of running tasks. Some are deployed as notebooks, as stand alone scripts, some can get data from Rucio. The data download part often needs to be separated from the compute for metrics to be understood better. We need to develop a proper schema for how the repo will be structured to accommodate all these options.

      Show
      The SRC workloads Gitlab repository needs to accommodate various ways of running tasks. Some are deployed as notebooks, as stand alone scripts, some can get data from Rucio. The data download part often needs to be separated from the compute for metrics to be understood better. We need to develop a proper schema for how the repo will be structured to accommodate all these options.
    • Hide
      • Restructure the repo so each of the tasks has clear ways of being run. The main categories are:
        • Run interactively in a Jupyter Notebook.
        • Run as a stand alone script with no user interaction.
        • Have the option to choose the data download methodology, such that it can be skipped, automated, or done by hand through Rucio.
        • Separate Docker pull from Docker run in the run.sh script so metrics are better calculated.
      Show
      Restructure the repo so each of the tasks has clear ways of being run. The main categories are: Run interactively in a Jupyter Notebook. Run as a stand alone script with no user interaction. Have the option to choose the data download methodology, such that it can be skipped, automated, or done by hand through Rucio. Separate Docker pull from Docker run in the run.sh script so metrics are better calculated.
    • 1
    • 1
    • 0
    • Team_MAGENTA
    • Sprint 5
    • Hide

      This was demoed as part of the STARS demo: https://drive.google.com/file/d/18_GJlVuDmbtjIK16z3Kqw38HXcVnrRUT/view?usp=drive_link 

      Show
      This was demoed as part of the STARS demo: https://drive.google.com/file/d/18_GJlVuDmbtjIK16z3Kqw38HXcVnrRUT/view?usp=drive_link  
    • 22.5
    • Stories Completed, Outcomes Reviewed, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO

    • PI23 - UNCOVERED

    • PI22 example-workflows-and-benchmarks

    Description

      • SRC workloads Gitlab repo admin - the way tasks are run needs to change to accommodate jupyter notebooks, metrics, benchmarking scores, and how we are integrating Rucio into some of them. We need to do an overhaul of the different ways tasks can be run.

      This can be done as other features are developed, but also this is something that will take some time and thinking, so it may need its own feature, or at least time set aside each sprint for.

      Attachments

        Issue Links

          mentioned in

          Page Loading...

          Structure

            Activity

              People

                P.Llopis Llopis, Pablo
                A.Clarke Clarke, Alex
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (85.71%)

                  Feature Estimate: 1.0

                  IssuesStory Points
                  To Do11.0
                  In Progress   00.0
                  Complete36.0
                  Total47.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel