Fully generic receive Helm chart

All SDP real-time workflows will have a very similar structure:

• They will be allocated on specific nodes (the addresses of which we want to transmit to CSP),
• want to write to storage, and
• might have many real-time processing (flagging, realtime calibration, fast imaging, metrics...) which we might want to feed via in-memory data exchange (e.g. Apache Plasma).

Therefore we should have a generic deployment method (i.e. Helm chart) that we only need to configure for the appropriate type of receive. This should reduce overall maintenance effort, and especially reduce how much workflow developers get exposed to Kubernetes internals.

• Standard Helm chart can now additionally instantiate the following when configured:
  • Plasma store (with configurable memory allocation) mounted into receive container
  • Arbitrary number of real-time processing containers connected to the store
• Support new receive chart explicitly in workflow library, provide documentation for how it is to be used (e.g. how to build a suitable container image).
• Port all current receive workflows to the new Helm chart

A new version (0.2.0) of the receive Helm chart has been developed as part of this feature. It is able to deploy a receive process, and optionally the plasma store and an arbitrary number of other containers as "processors" to consume data from the store.

https://gitlab.com/ska-telescope/sdp/ska-sdp-helmdeploy-charts/-/tree/master/charts/receive

The vis_receive workflow has been updated to use the chart. The new version is 0.3.6. It is capable of deploying the receiver and measurement set writer developed by the Yanda team with the plasma store.

https://gitlab.com/ska-telescope/sdp/ska-sdp-science-pipelines/-/tree/master/src/vis_receive

Older visibility receive workflows (plasma_pipeline) and charts (cbf-sdp-emulator, cbf-multicast-example, plasma-pipeline) have been retained for backward compatibility for the time being, but they will be deprecated.