SRC workflow example: e-MERLIN pipeline

One of the most important aspects of the science platform will be the provision of workflows, which will include providing the ability for users/teams to construct their own workflows for their data analysis as well as being able to run standard workflows such as imaging and calibration pipelines for the SKA and its precursors. The benefit of this feature is to build upon the work done in SP-3286, and take a step towards the provision of workflows for the full SDH&P pipelines, by implementing the e-MERLIN pipeline in the language of a workflow-execution engine and demonstrate it running on SRC resources.

AC1: A full set of scripts for the e-MERLIN pipeline, including CASA scripts and scripts written in the language of an appropriate workflow-execution engine, stored in git, in or linked from the SRC Workflows repository

AC2: An example supporting dataset made available to anybody within SRCNet, and stored somewhere, with links in jira.

AC3: The e-MERLIN pipeline demonstrated to be working on SRC resources, using the workflow-execution engine.

AC1: The Nextflow script can be found in the Tangerine github at https://gitlab.com/srcnet-tangerine/nextflow-prototyping/nextflow-scripts

The CASA scripts can be found in the github at https://gitlab.com/srcnet-tangerine/nextflow-prototyping/casa-scripts/-/tree/main/e-merlin-pipeline?ref_type=heads

The recipe file for the Singularity container can be found in github at https://gitlab.com/srcnet-tangerine/nextflow-prototyping/singularity-recipes

AC2: The dataset is currently stored on spsrc17.iaa.csic.es at ~/e-merlin/data/. Discussions are taking place regarding a more permanent archive for these data.

AC3: The Nextflow version of the e-MERLIN pipeline has been demonstrated to the ART on 24/08/2023.