LOW CSP test report

Summary

1
1 passed

Tests

tests/integration/test_correlator_observation_readiness.py 1 0:03:21.114381

PASSED test_correlator_observation_readiness 0:03:21.114381

This test performs a single observation on a single LOW-CSP subarray. It uses a simulated LFAA input signal containing a CW data source for station beams 1 through 6. The CW data source contains a tone at 312.5 MHz and is spread out across 8 coarse channels in the range 397 through 404, where coarse channel 400 corresponds to the center of the tone.

The subarray is configured to perform a correlation on the 6 station beams for all 8 coarse channels through LOW-CBF. It uses a simulated delay polynomial source where all delays are fixed at 0.

The input data is replayed from a PCAP file using a CNIC device, and is routed to the appropriate LOW-CBF correlator through the P4 switch.

The SPEAD output from the LOW-CBF correlator is captured in a PCAP file using a CNIC device. It is expected to contain output for 21 baselines across 1152 fine channels. The test also verifies whether the frequency range covered by the 1152 fine channels matches the frequency range covered by the 8 coarse channels in the input signal. These properties are verified by extracting the SPEAD metadata from the PCAP file.

Reference: CLT-9.

Setup

Captured log setup
2023-12-22 09:27:55  Allocators: ['low-cbf/allocator/0']
2023-12-22 09:27:55  CBF Controller devices: ['low-cbf/control/0']
2023-12-22 09:27:55  CBF Subarray devices: ['low-cbf/subarray/01', 'low-cbf/subarray/02', 'low-cbf/subarray/03', 'low-cbf/subarray/04']
2023-12-22 09:27:55  Connectors: ['low-cbf/connector/0']
2023-12-22 09:27:55  Delay devices: ['low-cbf/delaypoly/0']
2023-12-22 09:27:55  CNICs: ['low-cbf/cnic/1', 'low-cbf/cnic/2']
2023-12-22 09:27:55  Processors: ['low-cbf/processor/0.0.0', 'low-cbf/processor/0.0.1', 'low-cbf/processor/0.0.2', 'low-cbf/processor/0.0.3']
2023-12-22 09:27:55  CSP Controller devices: ['low-csp/control/0']
2023-12-22 09:27:55  CSP Subarray devices: ['low-csp/subarray/01', 'low-csp/subarray/02', 'low-csp/subarray/03', 'low-csp/subarray/04']

Call

Captured log call
2023-12-22 09:27:56  Initializing CNIC low-cbf/cnic/2
2023-12-22 09:28:22  Initializing CNIC low-cbf/cnic/1
2023-12-22 09:29:11  Tx: port 13/0, S/N XFL1XXQM0FKW, CnicDevice(low-cbf/cnic/1)
2023-12-22 09:29:11  XF: port  5/0, S/N XFL1IYUNES2E, LowCbfProcessor(low-cbf/processor/0.0.0)
2023-12-22 09:29:11  Rx: port  9/0, S/N XFL1EG4H5YXY, CnicDevice(low-cbf/cnic/2)
2023-12-22 09:29:11  Caught exception while trying to configure PTP Clock: bfrt_grpc.client.BfruntimeReadWriteRpcException: Error(s):
	* At index 0: ALREADY_EXISTS, 'Table Add failed table:pipe.ing_port Already exists'; Failed to execute command_inout on device low-cbf/connector/0, command AddPTPClockPort
2023-12-22 09:29:11  Routing table: {"Basic": [{"ingress port": "13/0", "port": "5/0"}, {"ingress port": "5/0", "port": "9/0"}]}
2023-12-22 09:29:11  P4 port status:
Port:  1/0  Enable: True   Up: True   Speed: 10G    Rx:    17669  Tx:        0
Port:  5/0  Enable: True   Up: True   Speed: 100G   Rx:      193  Tx:    17271
Port:  9/0  Enable: True   Up: True   Speed: 100G   Rx:      195  Tx:    29943
Port: 13/0  Enable: True   Up: True   Speed: 100G   Rx:    12673  Tx:   212961
Port: 17/0  Enable: True   Up: True   Speed: 100G   Rx:      189  Tx:    17271
Port: 21/0  Enable: True   Up: True   Speed: 100G   Rx:   196034  Tx:    17271
Port: 25/0  Enable: True   Up: True   Speed: 100G   Rx:      171  Tx:    17271
2023-12-22 09:29:11  Configuring low-cbf/cnic/2 for Rx: {'method': 'receive_pcap', 'arguments': {'out_filename': '/test-data/pcap/output/LFAATestPCAP_cw_8ch_10s.pcap', 'packet_size': 80, 'n_packets': 12672}}
2023-12-22 09:29:11  Turning off TANGO device: low-csp/subarray/01
2023-12-22 09:29:11  Turning off TANGO device: low-csp/control/0
2023-12-22 09:29:11  Waiting for TANGO device to turn off: low-csp/subarray/01
2023-12-22 09:29:11  Waiting for TANGO device to turn off: low-csp/control/0
2023-12-22 09:29:11  Initializing TANGO device: low-cbf/control/0
2023-12-22 09:29:11  Initializing TANGO device: low-csp/control/0
2023-12-22 09:29:12  Initializing TANGO device: low-cbf/subarray/01
2023-12-22 09:29:12  Initializing TANGO device: low-csp/subarray/01
2023-12-22 09:29:12  Waiting for TANGO device to respond: low-cbf/control/0
2023-12-22 09:29:12  Waiting for TANGO device to respond: low-csp/control/0
2023-12-22 09:29:12  Waiting for TANGO device to respond: low-cbf/subarray/01
2023-12-22 09:29:12  Waiting for TANGO device to respond: low-csp/subarray/01
2023-12-22 09:29:12  Turning on TANGO device: low-csp/subarray/01
2023-12-22 09:29:12  Turning on TANGO device: low-csp/control/0
2023-12-22 09:29:12  Waiting for TANGO device to start communicating: low-csp/subarray/01
2023-12-22 09:29:16  Waiting for TANGO device to start communicating: low-csp/control/0
2023-12-22 09:29:16  low-csp/control/0 in ON: ON
2023-12-22 09:29:16  low-csp/control/0 in healthState.UNKNOWN: healthState.UNKNOWN
2023-12-22 09:29:16  low-csp/control/0 in adminMode.ONLINE: adminMode.ONLINE
2023-12-22 09:29:16  low-csp/subarray/01 in ON: ON
2023-12-22 09:29:16  low-csp/subarray/01 in healthState.UNKNOWN: healthState.UNKNOWN
2023-12-22 09:29:16  low-csp/subarray/01 in adminMode.ONLINE: adminMode.ONLINE
2023-12-22 09:29:16  Assign resources: {'interface': 'https://schema.skao.int/ska-low-csp-assignresources/2.0', 'common': {'subarray_id': 1}, 'lowcbf': {}}
2023-12-22 09:29:16  Waiting for subarray to become IDLE
2023-12-22 09:29:18  low-csp/subarray/01 in obsState.IDLE
2023-12-22 09:29:18  Found XFL1IYUNES2E in FSP 3
2023-12-22 09:29:18  Configure scan: {'interface': 'https://schema.skao.int/ska-low-csp-configurescan/0.0', 'subarray': {'subarray_name': 'science period 23'}, 'common': {'config_id': 'sbi-mvp01-20200325-00001-science_A', 'subarray_id': 1, 'frequency_band': 'low'}, 'lowcbf': {'stations': {'stns': [[1, 1], [2, 1], [3, 1], [4, 1], [5, 1], [6, 1]], 'stn_beams': [{'beam_id': 1, 'freq_ids': [397, 398, 399, 400, 401, 402, 403, 404], 'delay_poly': 'low-cbf/delaypoly/0/delay_s01_b01'}]}, 'vis': {'fsp': {'firmware': 'vis:0.0.3', 'fsp_ids': [3]}, 'stn_beams': [{'stn_beam_id': 1, 'host': [[0, '192.168.0.1']], 'mac': [[0, '0c-42-a1-9c-a2-1b']], 'port': [[0, 20000, 1]], 'integration_ms': 849}]}}}
2023-12-22 09:29:18  Waiting for subarray to become READY
2023-12-22 09:29:20  Waiting for correlator personality to program
2023-12-22 09:29:20  Backing off assert_correlator_ready(...) for 0.0s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:20  Backing off assert_correlator_ready(...) for 0.0s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:20  Backing off assert_correlator_ready(...) for 0.2s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:20  Backing off assert_correlator_ready(...) for 0.4s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:21  Backing off assert_correlator_ready(...) for 0.2s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:21  Backing off assert_correlator_ready(...) for 0.9s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:22  Backing off assert_correlator_ready(...) for 0.8s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:23  Backing off assert_correlator_ready(...) for 0.5s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:23  Backing off assert_correlator_ready(...) for 2.6s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:26  Backing off assert_correlator_ready(...) for 9.8s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:35  Backing off assert_correlator_ready(...) for 3.2s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:29:39  Backing off assert_correlator_ready(...) for 41.4s (AssertionError: assert False is True
 +  where False = <built-in method get of dict object at 0x7f5b44c7d700>('ready', False)
 +    where <built-in method get of dict object at 0x7f5b44c7d700> = {'firmware': 'vis:0.0.3', 'ready': False}.get)
2023-12-22 09:30:20  Turning on correlator
2023-12-22 09:30:22  Resetting correlator
2023-12-22 09:30:22  Packet counts after Configure(): expected [(0, 0), (0, 0), (0, 0)], actual [(16, 1644), (5, 1475), (23, 1649)]
2023-12-22 09:30:22  Start scan: {'interface': 'https://schema.skao.int/ska-low-csp-scan/0.0', 'common': {'subarray_id': 1}, 'lowcbf': {'scan_id': 12345}}
2023-12-22 09:30:22  Waiting for subarray to become SCANNING
2023-12-22 09:30:24  After scan command the ports are: expected [(0, 0), (1152, 0), (0, 1152)], actual [(16, 1701), (1157, 1532), (23, 2858)]
2023-12-22 09:30:24  Preparing Tx on low-cbf/cnic/1: {'method': 'prepare_transmit', 'arguments': {'in_filename': '/test-data/pcap/input/LFAATestPCAP_cw_8ch_10s.pcap', 'rate': 1.44}}
2023-12-22 09:30:59  Transmitting from low-cbf/cnic/1: {'method': 'begin_transmit'}
2023-12-22 09:31:09  After scan command the ports are: expected [(195840, 0), (11520, 195840), (0, 11520)], actual [(195867, 2728), (11525, 198410), (35, 14253)]
2023-12-22 09:31:09  Ending scan
2023-12-22 09:31:09  Waiting for subarray to become READY
2023-12-22 09:31:11  Going to idle
2023-12-22 09:31:11  Waiting for subarray to become IDLE
2023-12-22 09:31:13  Releasing resources
2023-12-22 09:31:13  Waiting for subarray to become EMPTY
2023-12-22 09:31:15  After scan command the ports are: expected [(195840, 0), (12672, 195840), (0, 12672)], actual [(195869, 2868), (12677, 198552), (36, 15545)]
2023-12-22 09:31:15  Tx: port 13/0, S/N XFL1XXQM0FKW, CnicDevice(low-cbf/cnic/1)
2023-12-22 09:31:15  XF: port 5/0, S/N XFL1IYUNES2E, LowCbfProcessor(low-cbf/processor/0.0.0)
2023-12-22 09:31:15  Rx: port 9/0, S/N XFL1EG4H5YXY, CnicDevice(low-cbf/cnic/2)
2023-12-22 09:31:15  Routing table: {"Basic": [{"ingress port": "13/0", "port": "5/0"}, {"ingress port": "5/0", "port": "9/0"}]}
2023-12-22 09:31:15  P4 port status:
Port:  1/0  Enable: True   Up: True   Speed: 10G    Rx:     2946  Tx:        0
Port:  5/0  Enable: True   Up: True   Speed: 100G   Rx:    12677  Tx:   198564
Port:  9/0  Enable: True   Up: True   Speed: 100G   Rx:       36  Tx:    15557
Port: 13/0  Enable: True   Up: True   Speed: 100G   Rx:   195869  Tx:     2880
Port: 17/0  Enable: True   Up: True   Speed: 100G   Rx:       30  Tx:     2880
Port: 21/0  Enable: True   Up: True   Speed: 100G   Rx:       27  Tx:     2880
Port: 25/0  Enable: True   Up: True   Speed: 100G   Rx:       32  Tx:     2880
2023-12-22 09:31:15  Start reading SPEAD data from file: /mnt/clp-k8s-fpga-worker-1/pcap/output/LFAATestPCAP_cw_8ch_10s.pcap
2023-12-22 09:31:16  Finished reading SPEAD data from file: /mnt/clp-k8s-fpga-worker-1/pcap/output/LFAATestPCAP_cw_8ch_10s.pcap

Teardown