Enable accessibility and management of PST data products through the SDP common data-product user interface

Reduce risk of data loss due to volatile local storage failure on PST recording servers. Automated transfer of PST data products reduces risk of data loss and mislabelling during manual transfers Optimal usage of PST local storage is achieved by automatically transferring data products as soon as the scan has concluded. PST data products have an ADR-55 compliant meta-data description, thereby improving data provenance. Accessibility of PST data products improved via search/download capabilities of the existing SDP data product dashboard.

PST Voltage recorder data products are decorated with a YAML file that includes the meta-data described in AT3-398, which is compliant with ADR-55 . Data products are automatically transferred to shared storage that is accessible to commissioning scientists via the SDP data product dashboard Automatic data transfers commence shortly after a Scan has concluded and the ADR-55 compliant YAML file has been locally generated. The transferred data product is visible in the SDP data product dashboard after the transfer has been completed.

PST file transfer functionality described using Gherkin   PST Voltage Recorder Data Products documented in Solution Intent PST AA0.5 Architecture updated in Solution Intent PST Deployment Architecture for Testing documented in Software Engineering PST Data File Transfer Plan documented in Software Engineering PST Voltage Recorder Data Product Description documented in Software Engineering Execution block identifier added to common block of CSP scan configuration schema New gitlab repository for ska-pst-send created and configured for python development New documentation for ska-pst-send created and connected to the SKA Developer Portal New application written that writes the Voltage Recorder Data Product Description for each scan computes statistics for each output data file transfers all Voltage Recorder data products from local storage to shared/persistent storage updates the Data Product Dashboard Data Product Dashboard and shared/persistent storage simulator integrated into PST test architecture many thanks to the Naledi team for collaborating with us on integration and fixing the bugs that were discovered along the way PST file transfer functionality demonstrated at the PI19 Demo ( slides )