Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3597

Make pulsar acceleration-search Fourier spectra available to the FPGA FDAS component

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Enabler
    • Must have
    • PI20
    • COM PSS SW
    • None
    • Data Processing
    • Hide

      Enables frequency-domain pulsar-candidate detection in the PSS pipeline (Cheetah) by implementing the interface to the generated Fourier spectra from the de-dispersed time-series

      Show
      Enables frequency-domain pulsar-candidate detection in the PSS pipeline (Cheetah) by implementing the interface to the generated Fourier spectra from the de-dispersed time-series
    • Hide

      • For multiple DM trials, the FPGA FDAS component will read the memory location of each Fourier spectrum for that trial, and process it sequentially.
      • For each memory location provided, FPGA FDAS must be able to correctly read the corresponding Fourier-spectrum data and process each spectrum without errors

      Show
      • For multiple DM trials, the FPGA FDAS component will read the memory location of each Fourier spectrum for that trial, and process it sequentially. • For each memory location provided, FPGA FDAS must be able to correctly read the corresponding Fourier-spectrum data and process each spectrum without errors
    • 2
    • 2
    • 0
    • Team_PSS
    • Sprint 3
    • Hide

      In summary, in this feature we have installed an Agilex FPGA card in a host machine at JBO, and executed the FDAS code multiple times consecutively without error. We have also implemented a host code that loads a collection of power spectra from a set of files, then sequentially executes the FDAS algorithm on the FPGA for each of the available spectra. This host code is available here: https://gitlab.com/ska-telescope/pss/ska-pss-fdas-fpga-interface

      In addition, we have created a number of test vectors containing bright pulsars with varying parameters (a sub-set of the "FDAS FOP " set as described in section 2.1 here: https://www.overleaf.com/read/tphgctvcvdkw#11a155), dedispersed them at the pulsar DM, converted them to Fourier power spectrum, and successfully searched them with FDAS FPGA. All results are available here: https://drive.google.com/drive/folders/1t8mOA3w5L9K1ak3yrtegzw0J_fFY1KhL and these results were also demonstrated as part of DP ART demo 20.3: https://confluence.skatelescope.org/display/SE/2023-10-18+DP+ART+System+Demo+20.3

      More detail is given below in the outcomes for each story:-

      ==========================================

      AT4-1120:

      The Agilex FPGA card was successfully installed at JBO. A detailed description of how to install the card the required positions of the board switches can be found in the comments oof the AT4-1120 jira ticket.

      ==========================================

      AT4-1132:

      The fdas2 program was exercised over a long period of time to verify the robustness of the entire system. The purpose was to monitor the interaction of the FPGA board inserted in one of the JBO machine PCIe slots. During the execution of the program it was monitored for CPU load, memory load, and any unexpected behaviour of the computer. There were no problems running the program multiple times.

      ==========================================

      AT4-1133:

      A set of diverse test vectors were selected to test the FDAS FPGA and provide results that could be easily identified and differentiated when testing in a pipeline where the different test vectors would be serially processed "back-to-back".

      ==========================================

      AT4-1134:

      A set of test vector files (.tim) files were successfully generated to support the testing of the FDAS FPGA. Each test vector had a single pulsar with a known frequency and orbital acceleration.

      ==========================================

      AT4-1135:

      Test vector observation files (.tim) files generated in story AT4-1134 were successfully Fourier Transformed with normalisation to the frequency domain using a MATLAB script mimicking the operation of the Complex Fourier Transform (CXFT)function. The resulting spectra can be found on the PSS Team Google Drive:-

      PSS Team/FDAS_AGILEX_PI20_RELEASE/AT4-1135_MATLAB_CXFT_OF_TIM_FILES

      https://drive.google.com/drive/folders/1SV7vutao1z-9iZrl34Dwpo5rOQ2eJsZ1

      There are the following sub-folders:-

      MATLAB_CXFT_SCRIPT: This contains the MATLAB script that performs the Fourier Transform with normalisation

      TIM_FILES: This contains the test vector ".tim" files in the time domain that are to be Fourier Transformed

      SPECTRUM_FOR_FDAS: This contains the output ".dat" files from the Fourier Transform which are the frequency spectra that the FDAS FPGA processes.

      ==========================================

      AT4-1136:

      The 18 different spectrum files from AT4-1135 were successfully processed by the FDAS FPGA. The FDAS FPGA output files were manually sifted to identify the pulsar candidates. In all 18 cases the pulsar was successfully identified by the FDAS FPGA with correct frequency and orbital acceleration.

      The results of the FDAS processing can be found on the PSS Team Google Drive:- 

      PSS Team/FDAS_AGILEX_PI20_RELEASE/AT4-1136_ANALYSIS_OF_FDAS_CANDIDATES_FROM_TIM_FILES

      https://drive.google.com/drive/folders/1t8mOA3w5L9K1ak3yrtegzw0J_fFY1KhL

      There is a README.txt file that explains all the different files in this folder. The FDAS sifted results file is:-

      MANUAL_SIFTED_RESULT_SUMMARY_WITH_AT4-1135_config1_sum1_global_4_3.txt

      This file has the sifted FDAS results for each of the 18 test vectors.  An example of the test vector naming convention is:-

      FDAS-FOP-MID_71ca21d_200.0_0.05_1.0_78.0_Gaussian_50.0_123123123

      -Pulsar Frequency = 200.0Hz

      -Pulse width/pulsar period = 0.05

      -DM Measure correction  = 1.0 electron per cm3

      -Pulsar  acceleration = 78.0 m/s/s ( means towards the observer)

      -Signal/Noise Ratio = 50.0

      This file shows that the FDAS FPGA successfully found each pulsar at the expected frequency and orbital acceleration.

      A PI 20 demonstration of this work was also provided.

      These pulsar candidates and the FDAS results can now be used to prove out the operation of the FDAS FPGA when it is tested with software that will form the basis of the integration into the Cheetah PSS pipeline.

      ==========================================

      AT4-1137:

      Implemented a host code that loads a collection of Dispersion Measurements from a set of files, then sequentially executes the FDAS algorithm on the FPGA for each of the available Dispersion Measurements, and finally fetches a list of candidates from FPGA system memory. The host code is a program written in C++ that makes use of an API that exposes functions to interact with the FDAS-FPGA code.

      Show
      In summary, in this feature we have installed an Agilex FPGA card in a host machine at JBO, and executed the FDAS code multiple times consecutively without error. We have also implemented a host code that loads a collection of power spectra from a set of files, then sequentially executes the FDAS algorithm on the FPGA for each of the available spectra. This host code is available here: https://gitlab.com/ska-telescope/pss/ska-pss-fdas-fpga-interface In addition, we have created a number of test vectors containing bright pulsars with varying parameters (a sub-set of the "FDAS FOP " set as described in section 2.1 here: https://www.overleaf.com/read/tphgctvcvdkw#11a155 ), dedispersed them at the pulsar DM, converted them to Fourier power spectrum, and successfully searched them with FDAS FPGA. All results are available here: https://drive.google.com/drive/folders/1t8mOA3w5L9K1ak3yrtegzw0J_fFY1KhL and these results were also demonstrated as part of DP ART demo 20.3: https://confluence.skatelescope.org/display/SE/2023-10-18+DP+ART+System+Demo+20.3 More detail is given below in the outcomes for each story:- ========================================== AT4-1120: The Agilex FPGA card was successfully installed at JBO. A detailed description of how to install the card the required positions of the board switches can be found in the comments oof the AT4-1120 jira ticket. ========================================== AT4-1132: The fdas2 program was exercised over a long period of time to verify the robustness of the entire system. The purpose was to monitor the interaction of the FPGA board inserted in one of the JBO machine PCIe slots. During the execution of the program it was monitored for CPU load, memory load, and any unexpected behaviour of the computer. There were no problems running the program multiple times. ========================================== AT4-1133: A set of diverse test vectors were selected to test the FDAS FPGA and provide results that could be easily identified and differentiated when testing in a pipeline where the different test vectors would be serially processed "back-to-back". ========================================== AT4-1134: A set of test vector files (.tim) files were successfully generated to support the testing of the FDAS FPGA. Each test vector had a single pulsar with a known frequency and orbital acceleration. ========================================== AT4-1135: Test vector observation files (.tim) files generated in story AT4-1134 were successfully Fourier Transformed with normalisation to the frequency domain using a MATLAB script mimicking the operation of the Complex Fourier Transform (CXFT)function. The resulting spectra can be found on the PSS Team Google Drive:- PSS Team/FDAS_AGILEX_PI20_RELEASE/AT4-1135_MATLAB_CXFT_OF_TIM_FILES https://drive.google.com/drive/folders/1SV7vutao1z-9iZrl34Dwpo5rOQ2eJsZ1 There are the following sub-folders:- MATLAB_CXFT_SCRIPT: This contains the MATLAB script that performs the Fourier Transform with normalisation TIM_FILES: This contains the test vector ".tim" files in the time domain that are to be Fourier Transformed SPECTRUM_FOR_FDAS: This contains the output ".dat" files from the Fourier Transform which are the frequency spectra that the FDAS FPGA processes. ========================================== AT4-1136: The 18 different spectrum files from AT4-1135 were successfully processed by the FDAS FPGA. The FDAS FPGA output files were manually sifted to identify the pulsar candidates. In all 18 cases the pulsar was successfully identified by the FDAS FPGA with correct frequency and orbital acceleration. The results of the FDAS processing can be found on the PSS Team Google Drive:-  PSS Team/FDAS_AGILEX_PI20_RELEASE/AT4-1136_ANALYSIS_OF_FDAS_CANDIDATES_FROM_TIM_FILES https://drive.google.com/drive/folders/1t8mOA3w5L9K1ak3yrtegzw0J_fFY1KhL There is a README.txt file that explains all the different files in this folder. The FDAS sifted results file is:- MANUAL_SIFTED_RESULT_SUMMARY_WITH_AT4-1135_config1_sum1_global_4_3.txt This file has the sifted FDAS results for each of the 18 test vectors.  An example of the test vector naming convention is:- FDAS-FOP-MID_71ca21d_200.0_0.05_1.0_78.0_Gaussian_50.0_123123123 -Pulsar Frequency = 200.0Hz -Pulse width/pulsar period = 0.05 -DM Measure correction  = 1.0 electron per cm3 -Pulsar  acceleration = 78.0 m/s/s ( means towards the observer) -Signal/Noise Ratio = 50.0 This file shows that the FDAS FPGA successfully found each pulsar at the expected frequency and orbital acceleration. A PI 20 demonstration of this work was also provided. These pulsar candidates and the FDAS results can now be used to prove out the operation of the FDAS FPGA when it is tested with software that will form the basis of the integration into the Cheetah PSS pipeline. ========================================== AT4-1137: Implemented a host code that loads a collection of Dispersion Measurements from a set of files, then sequentially executes the FDAS algorithm on the FPGA for each of the available Dispersion Measurements, and finally fetches a list of candidates from FPGA system memory. The host code is a program written in C++ that makes use of an API that exposes functions to interact with the FDAS-FPGA code.
    • 20.4
    • Stories Completed, Outcomes Reviewed, NFRS met, Demonstrated, Satisfies Acceptance Criteria, Accepted by FO
    • PI22 - UNCOVERED

    • PSS-G1

    Description

      This is part of the work to integrate the FPGA FDAS component into Cheetah. It provides access to the memory locations of the FFT spectra for each DM trial.

      Attachments

        Issue Links

          Structure

            Activity

              People

                A.Noutsos Noutsos, Aristeidis
                L.Levin-Preston Levin-Preston, Lina
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 2.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete722.0
                  Total722.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel