Visibility memory data model

Why now? We will require common (memory) data models to exchange visibility data between realtime pipelines, likely by next PI

Scope: Data Processing realtime (and eventually likely batch) processing blocks

Quality: Modifiability, Testability, Performance, Portability, Scalability

Visibilities are one of the main types of data we will need to exchange within SDP pipelines. This makes it critical to find a good yet extensible way to represent them.

Fundamentally, visibilities are complex flux measurements in spatial frequency space. There are a couple of fundamental questions:

• How to represent their position? The "native" position scheme for ungridded visibilities should likely be (bl,t,ch,pol), but this needs to be mapped to indices. This might be especially tricky in the case of baseline-dependently averaged visibilities.
• How to represent data? Easiest option is to simply represent them as two complex numbers, but we likely want to support at least half, single and double precision here. Further compression (shared exponent, Dysco) is likely not appropriate for memory data models?
• (What types of polarisation to support? I.e. X/Y vs L/R? Do we need to decide this at this point?)

As a secondary concern, there is some meta data that we need to carry around to make sense of the visibilities:

• Baseline map, likely referencing an antenna table
• Time table?
• Frequency table?
• Polarisation table?
• Basic beam data (size of field of view)

Other visibility-associated data we might want to discuss:

• Weights?
• Flags & centroid offsets?