Scaling of SDP pipelines for AA*

At this point it seems likely that the components we have been using so far cannot be incrementally refactored into a solution that is scalable enough to carry us to AA* - at least not with our resources, and the pressure to support AA2 comissioning.

Therefore this epic aims to develop new pipeline architectures that prioritise scaling. Our primary focus is to revise how work and I/O is distributed, and this will require some new development. On the other hand, we aim to share as many components (specifically processing components) with AA2 pipelines as possible in order to not duplicate effort.

The goal for this epic is to first demonstrate scaling and performance goals (including support for accelerators) as early as possible, followed by adding critical functionality required for production-ready self-calibration pipeliens.