Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3508

SRC workflow example: Image Convolution

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      SRCs will provide pre-built scripts for standard workflows that users expect to run. We aim to implement some of these, and test their effectiveness with regards to hardware limitations and data storage schemes. 

      This particular workflow looks at convolving images to change the beam shape/size. This workflow could test distributing images across a storage network (Rucio), and running convolution on each of them simultaneously. 

      Show
      SRCs will provide pre-built scripts for standard workflows that users expect to run. We aim to implement some of these, and test their effectiveness with regards to hardware limitations and data storage schemes.  This particular workflow looks at convolving images to change the beam shape/size. This workflow could test distributing images across a storage network (Rucio), and running convolution on each of them simultaneously. 
    • Hide

      AC1: Run a containerised script that can change the image beam shape by convolving the image. 

      AC2: Use DASK to demonstrate more efficient workflow

      AC3: Add task to the Gitlab repo.

      Show
      AC1: Run a containerised script that can change the image beam shape by convolving the image.  AC2: Use DASK to demonstrate more efficient workflow AC3: Add task to the Gitlab repo.
    • 0.5
    • 0.5
    • 0
    • Team_CORAL
    • Sprint 5
    • Hide

      Documentation: https://confluence.skatelescope.org/display/SRCSC/COR-424%5Bworkflow-convolution%5D++Cod[…]he+Python+Script+for+convolutional+ops.+with+local+data

       

      Repo: https://gitlab.com/ska-telescope/src/src-workloads/-/tree/amendoza/tasks/img_gauss_conv?ref_type=heads

      Show
      Documentation: https://confluence.skatelescope.org/display/SRCSC/COR- 424 %5Bworkflow-convolution%5D++Cod[…]he+Python+Script+for+convolutional+ops.+with+local+data   Repo: https://gitlab.com/ska-telescope/src/src-workloads/-/tree/amendoza/tasks/img_gauss_conv?ref_type=heads
    • 24.3
    • Stories Completed, Outcomes Reviewed

    • PI24 - UNCOVERED

    • PI19-PB PI20 PI20-PB

    Description

      Input data: LOFAR images

      Output data: image at a lower resolution or altered beam shape

      Software involved: Python, astropy, Dask

      Workflow steps: Run convolution using astropy on the fits data. Pay attention to the convolution type and memory requirements to optimise the speed. Could run this on many images simultaneously, making use of distributing the data. Could use DASK to demonstrate more efficient compute.

      State of existing workflows: Not looked at but would be a basic small python script written from scratch. A small amount of work. Could then be combined with mosaicking/source-finding workflows. 

      Architecture and hardware: Multithreaded CPU. RAM could be important for large arrays. Data in a single location or distributed.

      Attachments

        Issue Links

          relates to

          Feature - A Feature is a service that fulfills a stakeholder need. Each feature includes a benefit hypothesis and acceptance criteria, and is sized or split as necessary to be delivered by a single Agile Release Train (ART) in a Program Increment (PI). SP-3496 Consolidate naming and structure of SRC example workflow repository

          • Must have - Items labelled as "Must have" are critical to the current delivery timebox in order for it to be a success.
          • Done
          mentioned in

          Page Loading...

          Structure

            Activity

              People

                A.Clarke Clarke, Alex
                r.bolton Bolton, Rosie
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 0.5

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete69.0
                  Total69.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel