Integrate the DDR Circuits developed for feature SP-2764 into the FDAS FPGA

The FDAS (Fourier Domain Acceleration Search) design loaded into the Intel Agilex FPGA (Field Programmable Gate Array) relies heavily on external DDR4 SDRAM memory (the same kind that is used in PCs/Laptops) to store large quantities of data, which the FDAS function accesses as-and-when needed via DDR4 (Double Data Rate version 4) interfaces that connect the FPGA to the SDRAM memory.

The Intel Agilex FPGA supports four DDR4 Interfaces and the Intel Agilex Development Board that we have purchased has a slot for each of these DDR4 interfaces to allow an SDRAM memory stick to be plugged in.

The Intel Agilex documentation states that one of these DDR4 SDRAM interfaces is normally assigned to support an internal “hard processor system” (HPS) within the FPGA. The larger modern FPGAs often include a small processor, however the FDAS design does not make use this function.

From the FDAS work carried out in PI15 for feature SP-2536 it was determined that this DDR4 SDRAM interface of the Intel Agilex FPGA that is normally assigned to an internal processor could be re-assigned to support the FDAS function.

The FDAS design that was translated from the Intel Arria 10 family to the Intel Agilex family in PI14 and PI15 currently only makes use of two DDR4 SDRAM interfaces, as that was all that that the Intel Arria 10 development boards could support. One DDR4 SDRAM interface is used to store the incoming observation data from the Host PC via the PCIe interface and the other DDR4 SDRAM interface is used to store the Filter Output Plane (FOP) data which is generated by the FDAS Convolution (CONV) module and subsequently interrogated by the FDAS Harmonic Summing (HSUM) module to identify possible pulsar candidates.

With the increased number of DDR4 SDRAM interfaces that the Intel Agilex FPGA and the Intel Agilex development board support it is possible to effectively combine together three DDR SDRAM interfaces to act in unison to effectively triple the bandwidth to access the FOP. The fourth DDR4 SDRAM interface will still support the storage of the incoming observation data via the PCIe Interface.

The CONV and HSUM modules have already been designed to take advantage of this increase in bandwidth to access the FOP via generic parameterisation (a technique that allows easy scaling of designs) which increases the width of the data bus to the DDR4 circuits.

In feature SP-2764 the DDR circuits were designed to support the “combining together” of three DDR4 SDRAM interfaces to connect to the CONV and HSUM modules.

In this feature the new DDR circuits shall be integrated into the FDAS design.

The tasks identified for this feature are:-

1. Update the FDAS Core Level structural VHDL to instantiate three new DDRIF2 modules to act in unison to provide an effective 1536-bit data interface for the Convolution (CONV) and Harmonic Summing (HSUM) modules to access the Filter Output Plane (FOP) stored in three external DDR SDRAM sticks.
2. Fix the VHDL parameter generic for the Convolution Load (CLD) module to 1, as this module is not affected by the “ddr_g” parameter generic that is applied to the CONV and HSUM modules. (The CLD, CONV and HSUM modules we all designed to support increased DDR SDRAM bandwidth access to the external DDR SDRAMs, however this feature is not currently required for the CLD module).
3. Update the PCIe Hard IP Macro to support DMA access to a total of four DDR Interfaces.
4. Update the FDAS Top Level structural VHDL to instantiate the new DDR_CONTROLLER_CALIBRATION, DDR_CONTROLLER_CALIBRATION_HPS and PCIe Hard IP modules/VHDL wrappers and connect appropriately to the FDAS Core and external FPGA pins for the DDR SDRAM interfaces.
5. Update the Intel Quartus .qsf file to include the changes for the new DDRIF2, DDR_CONTROLLER_CALIBRATION and DDR_CONTROLLER_CALIBRATION_HPS modules.
6. Build the new FDAS design in the Intel Quartus Prime software with the “ddr_g” VHDL parameter generic set to 3 so that the CONV and HSUM modules are built to use all three DDR SDRAM interfaces provided to store the FOP.
7. Analyse the result of the build, especially with regard to resource utilisation and maximum core clock frequency and fix any bugs.
8. Document the design in the FDAS Implementation specification and include the results of the build.
9. Copy the complete design to the PSS Google Drive.

Given:  That feature SP-2764 has been successfully completed and DDR Interface circuits are available to provide increased access bandwidth to the FDAS FPGA Filter Output Plane (FOP).

When: The new DDR Interface circuits have been integrated into the FDAS FPGA design.

Then: The updated FDAS design can be built via the Intel Quartus Prime Software and performance/ resource utilisation results can be obtained to guide the options for future developments, such as the inclusion of other functions within the FPGA to sit alongside the FDAS function.

The tasks completed in this feature are:-

1. Successfully updated the FDAS Core Level structural VHDL to instantiate three new DDRIF2 modules (designed in feature SP-2764) to act in unison to provide an effective 1536-bit data interface for the Convolution (CONV) and Harmonic Summing (HSUM) modules to access the Filter Output Plane (FOP) stored in three external DDR SDRAM sticks.
2. Fixed the VHDL parameter generic for the Convolution Load (CLD) module to 1, as this module is not affected by the “ddr_g” parameter generic that is applied to the CONV and HSUM modules. (The CLD, CONV and HSUM modules we all designed to support increased DDR SDRAM bandwidth access to the external DDR SDRAMs, however this feature is not currently required for the CLD module).
3. Successfully updated the PCIe Hard IP Macro to support DMA access to a total of four DDR Interfaces.
4. Successfully updated the FDAS Top Level structural VHDL to instantiate the new DDR_CONTROLLER_CALIBRATION, DDR_CONTROLLER_CALIBRATION_HPS (both designed in feature SP-2764) and PCIe Hard IP modules/VHDL wrappers and connect appropriately to the FDAS Core and external FPGA pins for the DDR SDRAM interfaces.
5. Successfully updated the Intel Quartus .qsf file to include the changes for the new DDRIF2, DDR_CONTROLLER_CALIBRATION and DDR_CONTROLLER_CALIBRATION_HPS modules.
6. Successfully built the new FDAS design in the Intel Quartus Prime software with the “ddr_g” VHDL parameter generic set to 3 so that the CONV and HSUM modules are built to use all three DDR SDRAM interfaces provided to store the FOP. The .qsf file SEED was set to 6 to produce a build that met all the clock frequency targets, most importantly the core clock target of 350MHz.
7. Analysed the result of the build, especially with regard to resource utilisation and maximum core clock frequency and fix any bugs. The build results were all documented in the FDAS Implementation specification.
8. Documented the design architecture in the FDAS Implementation specification and included the results of the build.
9. Copied the complete design to the PSS Google Drive in the following folders:-

Design Repository

FDAS_AGILEX_PI16_RELEASE/FDAS3_REPOSITORY

https://drive.google.com/drive/folders/13ml_rJbSsFPQ3iNKCREXKVjH_JG06Fkt

 
Build Environment

(files in correct folders for an Intel Quartus ver 22.2 build and also the results of the build).

FDAS_AGILEX_PI16_RELEASE/FDAS2_BUILD

https://drive.google.com/drive/folders/1oOFjwfYwIDKzT-DAgH9Az-EmUuuJBade

This version of the FDAS FPGA with three DDR SDRAM interfaces acting in unison to provide a higher bandwidth access to the Filter Output Plane (FOP) is now ready for testing in an Agilex FPGA.