7.3 KiB
+++ title = "A tale of working with Xilinx DisplayPort & UIO" date = 2019-08-30 +++
At work, a DisplayPort IP from Xilinx was being used. Xilinx doesn't provide any driver for this. There is a TX and RX.
Bare metal code support is provided, however, support was needed for Linux. Ignoring interrupts, it's easy to get this bare metal code to work on Linux. Xilinx's bare metal code at it's core uses Xil_Out32 and Xil_In32 function for writing and reading to registers. The implementations for these can be replaced with mmap for accessing the registers. For DP TX side, doesn't need to handle interrupts and setting up the registers is enough. For RX, however, interrupts are needed to setup RX. For example, the link training for DP is initiated once a Training Pattern 1 (TP1) interrupt is detected.
Linux being a monolithic kernel, there is clear separation between kernel and user space. Interrupts can only be handled in kernel space. However, it was easier to use the ported bare metal code in user space and so the need to handle interrupts in user space. One writes a driver to do all this but since only the interrupt part had to be handled in kernel space, the UIO subsystem was needed.
Using the UIO subsystem, it's possible to handle the interrupts in kernel space while the rest like reading or writing to the registers can be done in user space. There's a Userspace I/O Platform driver with generic IRQ handling code. Taking the example of Xilinx DisplayPort RX here. Colleague who works on FPGA side generates the FPGA firmware along with device trees which has entries as per the peripherals configured on FPGA side, for example, DP RX peripheral can be in the memory region 0x80004000 to 0x80006000. An interrupt is assigned based on how the FPGA Programmable Logic (PL) connects to the Processing System (PS). PL is the FPGA and PS is the ARM64 SoC.
The extended device tree entry looks like this. reg specifies the memory that can be mmaped in user space and accessed while the interrupts get used by the kernel code.
&SUBBLOCK_DP_BASE_v_dp_rxss1_0 {
compatible = "dprxss-uio";
interrupt-parent = <&gic>;
interrupts = <0 92 4 0 92 4>;
reg = <0x0 0x80004000 0x0 0x2000>
status = "okay";
}
To link the UIO platform driver to this, add the following to the bootargs environment variable in u-boot.
uio_pdrv_genirq.of_id=dprxss-uio
There is a need to do this, since the compatible property for device tree isn't specified in the driver. See here. So it's a module parameter.
static struct of_device_id uio_of_genirq_match[] = {
{ /* This is filled with module_parm */ },
{ /* Sentinel */ },
};
MODULE_DEVICE_TABLE(of, uio_of_genirq_match);
module_param_string(of_id, uio_of_genirq_match[0].compatible, 128, 0);
MODULE_PARM_DESC(of_id, "Openfirmware id of the device to be handled by uio");
Now, a combination of poll and read can be used to wait for interrupts in user space. So all well and good, however, there are some caveats to know. Once an interrupt is handled in kernel code, it disables the interrupt.
static irqreturn_t uio_pdrv_genirq_handler(int irq, struct uio_info *dev_info)
{
struct uio_pdrv_genirq_platdata *priv = dev_info->priv;
/* Just disable the interrupt in the interrupt controller, and
* remember the state so we can allow user space to enable it later.
*/
spin_lock(&priv->lock);
if (!__test_and_set_bit(UIO_IRQ_DISABLED, &priv->flags))
disable_irq_nosync(irq);
spin_unlock(&priv->lock);
return IRQ_HANDLED;
}
The interrupt re-enable logic is in the below function.
static int uio_pdrv_genirq_irqcontrol(struct uio_info *dev_info, s32 irq_on)
{
struct uio_pdrv_genirq_platdata *priv = dev_info->priv;
unsigned long flags;
/* Allow user space to enable and disable the interrupt
* in the interrupt controller, but keep track of the
* state to prevent per-irq depth damage.
*
* Serialize this operation to support multiple tasks and concurrency
* with irq handler on SMP systems.
*/
spin_lock_irqsave(&priv->lock, flags);
if (irq_on) {
if (__test_and_clear_bit(UIO_IRQ_DISABLED, &priv->flags))
enable_irq(dev_info->irq);
} else {
if (!__test_and_set_bit(UIO_IRQ_DISABLED, &priv->flags))
disable_irq_nosync(dev_info->irq);
}
spin_unlock_irqrestore(&priv->lock, flags);
return 0;
}
This is called from uio_write. And if you know how file operations work, this uio_write is called when a write system call is issued with a file descriptor received from opening the /dev/uioX node.
static const struct file_operations uio_fops = {
.owner = THIS_MODULE,
.open = uio_open,
.release = uio_release,
.read = uio_read,
.write = uio_write,
.mmap = uio_mmap,
.poll = uio_poll,
.fasync = uio_fasync,
.llseek = noop_llseek,
};
Now, here comes the problem. After the first interrupt, no more interrupts were being handled. So, basically the write call wasn't working to re-enable the interrupt. Putting print statements in uio_write, it could be seen that a call to the write function didn't result in invocation of uio_write.
After being perplexed and wasting 4-5 hours trying to figure out what's wrong, wrote a small piece of code outside the project workspace which opened /dev/uioX and then did a write. In this case, the prints from uio_write function which eventually called uio_pdrv_genirq_irqcontrol to enable the interrupt could be seen. So, something was wrong with the project setup.
Using neovim and ctags for code navigation, trying jump to definition on the write call, ended up in a write.c file. The first initial project setup was done by the FPGA engineer colleague since Xilinx SDK generates bare metal code samples based on the design done. Had not noticed this file before. It seemed to be an artifact of the code ported over from bare metal and had a write function as below.
__attribute__((weak)) sint32 write(sint32 fd, char8 *buf, sint32 nbytes)
Being aware of the weak attribute from working in u-boot where it's used to define board specific functions over riding default ones. The GCC manual defines it as The weak attribute causes the declaration to be emitted as a weak symbol rather than a global. This is primarily useful in defining library functions which can be overridden in user code.
There was no other write function defined in the project. Ideally it should have been picked up from glibc. However, this wasn't happening. Didn't need that write implementation in write.c which was actually writing to a UART port, so after removal everything started working fine. DP link training was succeeding finally.
One can read more about UIO here.