Reset FPGA based PCIe card and restore its Config Space

Question

I am adapting a Windows / Linux driver of a FPGA based PCIe card. (using a LatticeECP3 with PCIe Endpoint) I need to add a driver function to allow a host driven bitstream update of the FPGA without the need of rebooting the host afterwards.

I.e. after flashing I would need to reset the FPGA to let the bitstream be reloaded. This of course would lead to a loss of the Config Space settings of PCIe endpoint.

My first approach was to implement following steps:

1. Save the PCI config space of the PCIe.
2. Put the device into sleep mode via power management functions.
3. Wake up the device via power management functions. This would trigger an FPGA reset.
4. Restore the PCI Config Space.

My questions:

1. Q1: Is this a supported use case for PCIe devices running on Windows 7 / Linux host platforms anyway?
2. Q2: What are the appropriate key words to find documentation or coding examples about this use case.

Answer 1

On Widows, you need to "disable/enable" through Device Manager, but programmatically. As a result your PCIe device will be re-enumerated by Windows and the PCI cfg space will get the correct values (not necessarily the same as before "disable").

SetupDiChangeState() is your friend, it does exactly what you need. Take a look at DevCon sample in Windows Drivers Development kit. It shows how to code this (the sample shows a lot of nice stuff besides disable/enable, which I'm sure you will love :) ).

One thing you need be careful with: if after reflushing your PCIe endpoint requires larger memory resources, the system-wide balance of PCIe memory resources may need to be changed, in that case it is simplest to just reboot. In short, avoid resizing of memory resources as a result of re-flush.

On Linux, it is not that easy. User mode keywords are "Linux hotplug subsystem". Kernel mode keywords include pci_enable_device() and pci_disable_device(). Maybe this link will help: How can the linux kernel be forced to enumerate the PCI-e bus?