Details
-
Enabler
-
Not Assigned
-
None
-
Data Processing
-
-
- casacore-based applications can read data stored in plasma in the form of Arrow Tables without modifications.
- Stretch: Internal references to column on other tables are handled transparently.
-
2
-
2
-
0
-
Team_YANDA
-
Sprint 5
-
-
-
-
9.3
-
Stories Completed, Outcomes Reviewed, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO
Description
To keep in sync with times and keep the plasma storage manager gluing code relevant, we want it to gain Apache Arrow Table reading support. At the moment it supports Tensors of arbitrary shapes and different types (except String), but the visibility data model uses Arrow Tables heavily, which the plasma storage manager doesn't support yet.
In addition to the value of making existing casacore-applications compatible with the ADR-21 visibility data model, this work would also serve as an external consumer of this model, thus further validating it.
This initial support includes only arbitrary columns from arbitrary Tables; as a stretch we could include handling references up to some degree.