Details
-
Epic
-
Not Assigned
-
None
-
None
-
SDP AA2 pipelines
-
Data Processing
-
0
-
- Ability to do end-to-end processing of visibility data from storage with performance suitable for AA2 commissioning & operations.
- Ability to generate data products suitable for AA2 Science Verification.
Description
This will continue the development of the self-calibration pipelines and continue to scale to AA2. Will also include the development of all required functionality to enable full SDP processing for Science Verification. This focuses more on pipeline integration of all required end-to-end functionality (including e.g. instrumental calibration), with the expectation that these can be run outside of the SDP system.
The goal is to demonstrate that we can perform the entire end-to-end data reduction to get sufficiently high-quality data products in a production-like system within acceptable wall time limits. What precisely sufficient quality and acceptable time limits are is to be discussed with stakeholders, but our baseline assumption is that we are targetting the same quality as the Rapthor pipeline, and less than a day of compute time for a 4 hour observation.