AA0.5 EDA design and deployment

Re-examine and refine design for the Engineering Data Archive (EDA),  including the choice of technology and deployment strategy.

Why now ?

There is a need to define platform and software requirements for AA0.5, so that hardware can be procured in time, technology selected, installed and tested and custom software developed. Persistent EDA instances will be needed as soon as testing starts in the PSIs and System ITFs, and its architecture for AA0.5 deployment has to be finalised.

Scope:

The scope of this ADR is limited to the archiving and querying of TANGO attributes. 

Considering deployment of equipment and estimated data flows, determine how EDA will be deployed in AA0.5 in Mid and Low Telescope, and in the System ITFs.  (Assume that EDA will be deployed in the PSIs in the same manner as in the System ITFs.)

A large number of attributes will have to be archived in the EDA backend from geographically distributed TANGO systems, and a number of alternatives are technically possible to address this scenario in a distributed way.

AA0.5 deployment diagrams for MID and Low Telescopes and ITFs are available in Solution Intent:

https://confluence.skatelescope.org/display/SWSI/SKA+Software+Deployment+view+AA0.5

Deployment strategy and technologies to be used for EDA frontend and backend should be selected in light of the Learning from others series of lectures and in consultation with Thomas.Juerges.

Quality:

Availability: In the geographically distributed system, where each telescope has equipment distributed over two or more locations,  each instance of the EDA backend can be queried from any location used by the SKA Team. 

Performance: latency between different environments and data availability in the event of network failures needs to be quantified. In a nominal scenario where the network is available EDA parameters shall be available to the users in a matter of seconds. 

Performance: the storage size at AA0.5 should not be an issue, with the only exception of the LOW TCPF where the EDA backend might grow as much as 10TB /year/station according to the analysis presented by the MCCS team. This needs to be taken into account when estimating network requirements for data streaming or replication. In all other cases, estimates of the capacity needed, number of attributes, number of users and access/query rate can be based on experience with similar facilities and rough estimates of the SKA use.