Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3222

Ensure MS contains enough information for pointing calibration

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Enabler
    • Must have
    • PI18
    • COM SDP SW
    • None
    • Data Processing
    • Hide

      See Why? in description

      Show
      See Why? in description
    • Hide

      See What? in description

      Show
      See What? in description
    • Intra Program
    • 2
    • 2
    • 0
    • Team_YANDA
    • Sprint 5
    • Hide

      The feature has delivered the functionality to add the required OFFSETs into a measurement set when written. This is provided by an interface to Kafka queue that contains commanded and actual positions.

      it should be noted that even if there are future changes to the contents of the Kafka queue due to architecture or design changes - the work done here still would be able to add the SOURCE OFFSET to the measurement set as the various elements are separated by well defined interfaces.

       

      Some of the outcomes from the child tickets are listed here.

       

      • We added a method to MS Writer in realtime-receive-core to write to the pointing table - using a class representing a simplified row in the pointing table (see pointing.py)
      • We added a test to test_msutils.py to verify the pointing data gets written correctly, and it passes.
      • We updated the documentation with instructions relevant to M1 Mac. See docs/src/installation.rst.
      • Changes have been merged into main (merge request !76).
      • We created a tech-debt ticket for a problem we encountered. In the realtime-receive repositories, documentation and helper scripts need to be updated to cater to MacOS & M1 Macs. See YAN-1306.
      • Changes have been merged to the ska-sdp-realtime-receive-processors repository. See merge requests !33 and !34.
      • Antenna pointing information, including pointing offsets, are written to the pointing table in the measurement set.
      • Pointing data (provided by a Tango device) is read by a Kafka consumer from two topics: a topic for ACTUAL pointings and a topic for COMMANDED pointings.
      • We generate a pointing entry (numpy array) for each antenna at the cadence of the actual pointing, and we extract the corresponding commanded pointing at this timestamp via a cubic spline interpolation (the InterpolatedUnivariateSpline method from SciPy).
      • Rows of pointing data are produced in mswriter_pocesssor.py, then transferred to src/realtime/receive/core/msutils.py where it is written to the measurement set.
      • In the ms pointing table, commanded pointings are written to the TARGET column, actual pointings are written to the DIRECTION column, and the offset (ACTUAL minus COMMANDED) is written to the SOURCE_OFFSET column.
      Show
      The feature has delivered the functionality to add the required OFFSETs into a measurement set when written. This is provided by an interface to Kafka queue that contains commanded and actual positions. it should be noted that even if there are future changes to the contents of the Kafka queue due to architecture or design changes - the work done here still would be able to add the SOURCE OFFSET to the measurement set as the various elements are separated by well defined interfaces.   Some of the outcomes from the child tickets are listed here.   We added a method to MS Writer in realtime-receive-core to write to the pointing table - using a class representing a simplified row in the pointing table (see pointing.py ) We added a test to test_msutils.py to verify the pointing data gets written correctly, and it passes. We updated the documentation with instructions relevant to M1 Mac. See docs/src/installation.rst . Changes have been merged into main (merge request !76 ). We created a tech-debt ticket for a problem we encountered. In the realtime-receive repositories, documentation and helper scripts need to be updated to cater to MacOS & M1 Macs. See YAN-1306. Changes have been merged to the ska-sdp-realtime-receive-processors repository. See merge requests !33 and !34 . Antenna pointing information, including pointing offsets, are written to the pointing table in the measurement set. Pointing data (provided by a Tango device) is read by a Kafka consumer from two topics: a topic for ACTUAL pointings and a topic for COMMANDED pointings. We generate a pointing entry (numpy array) for each antenna at the cadence of the actual pointing, and we extract the corresponding commanded pointing at this timestamp via a cubic spline interpolation (the InterpolatedUnivariateSpline method from SciPy). Rows of pointing data are produced in mswriter_pocesssor.py , then transferred to src/realtime/receive/core/msutils.py where it is written to the measurement set. In the ms pointing table, commanded pointings are written to the TARGET column, actual pointings are written to the DIRECTION column, and the offset (ACTUAL minus COMMANDED) is written to the SOURCE_OFFSET column.
    • 22.3
    • Stories Completed, Integrated, Solution Intent Updated, BDD Testing Passes (no errors), Outcomes Reviewed, NFRS met, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO
    • PI24 - UNCOVERED

    • SDP-G1

    Description

      Following discussions at the Orca and Hippo architecture workshop, it was decided that the best way to integrate the pointing offset calibration pipeline is to use a measurement set as input which should contain the pointing information the pipeline needs to run. The pipeline runs after a reference pointing scan is completed and the pointing and visibility data have been written to the MS. See SDP Goal 1 for further information.

      Who?

      • SDP real-time pipeline developers and integrators.

      What?

      • When capturing data from a reference pointing scan, the visibility receive processing script writes pointing information into the measurement set alongside the visibilities.
      • Ideally we would write into the MS pointing table the "SOURCE_OFFSET" column in Az/El coordinates

      Why?

      • In order to integrate the pointing offset calibration pipeline, the measurement set must contain the pointing information needed by the pipeline.

      Attachments

        Issue Links

          Structure

            Activity

              People

                p.wortmann Wortmann, Peter
                m.ashdown Ashdown, Mark
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete510.0
                  Total510.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel