Initial prototype of RT RFI flagging pipeline functionality (considering interfaces that can be tested with the AA0.5 system)

The prototype of the pseudo-real-time pipeline was implemented using C++ and stored in the SKA repository. Input data format was chosen to be the measurement set from CASA as it is a standard, and simulated data from the SIM team was provided in Measurement set format. The RFI algorithm itself is an initial implementation of the sum threshold algorithm. The implementation can read input data in a measurement set format, perform RFI mitigation and update the input measurement set data with RFI flags. The data used for testing is a simulation of a Perth DTV transmitter. A basic form of tests was performed where each flagged visibility is checked with the RFI location, which the SIM team also provided. The current test shows that 30% of flags correctly flags RFI present in the input data, while 70% of flags were false positive. However, the RFI pattern is switched on and off randomly, and the sum threshold algorithm flags larger chunks of data which may cause this discrepancy. Since the RFI mitigation implemented contains only a sum threshold algorithm, it is expected that RFI mitigation will improve with more advanced methods. Repository: https://gitlab.com/ska-telescope/ska-post-correlation-rfi-flagger/-/tree/master RFI mitigation description: https://confluence.skatelescope.org/display/SE/Pseudo+Real-time+RFI+Flagging+Prototype+Design Summary of implemented RFI flagger: https://confluence.skatelescope.org/display/SE/Initial+Implementation+of+the+RFI+Flagger Data used: https://confluence.skatelescope.org/display/SE/RFI+simulations+for+the+Real-Time+pipeline Future work: Investigate resulting flags to identify the source of a large number of false-positive flags. Investigate simulated data and correct RFI flags provided by SIM for discrepancies Implement more advanced RFI mitigation algorithms