pynq.lib.video Module¶
The pynq.lib.video module is a driver capturing streaming HDMI input, producing streaming HDMI output and hardware-accelerated colorspace conversion.
pynq.lib.video.clocks Module¶
pynq.lib.video.common Module¶
-
class
pynq.lib.video.common.
PixelFormat
(bits_per_pixel, in_color, out_color, fourcc=None)[source]¶ Bases:
object
Wrapper for all of the information about a video format
-
bits_per_pixel
¶ Number of bits for each pixel
Type: int
-
in_color
¶ Coefficients from BGR stream to pixel format
Type: list of float
-
out_color
¶ Coefficient from pixel format to BGR stream
Type: list of float
-
-
class
pynq.lib.video.common.
VideoMode
(width, height, bits_per_pixel, fps=60, stride=None)[source]¶ Bases:
object
Class for holding the information about a video mode
-
height
¶ Height of the video frame in lines
Type: int
-
width
¶ Width of the video frame in pixels
Type: int
-
stride
¶ Width of a line in the video frame in bytes
Type: int
-
bits_per_pixel
¶ Bits per pixel
Type: int
-
bytes_per_Pixel
¶ Bytes required to represent each pixel
Type: int
-
shape
¶ Numpy-style tuple describing the video frame
Type: tuple of int
-
pynq.lib.video.dma Module¶
-
class
pynq.lib.video.dma.
AxiVDMA
(description, framecount=4)[source]¶ Bases:
pynq.overlay.DefaultIP
Driver class for the Xilinx VideoDMA IP core
The driver is split into input and output channels are exposed using the readchannel and writechannel attributes. Each channel has start and stop methods to control the data transfer. All channels MUST be stopped before reprogramming the bitstream or inconsistent behaviour may result.
The DMA uses a single ownership model of frames in that frames are either owned by the DMA or the user code but not both. S2MMChannel.readframe and MM2SChannel.newframe both return a frame to the user. It is the user’s responsibility to either free the frame using the freebuffer() method or to hand ownership back to the DMA using MM2SChannel.writeframe. Once ownership has been returned the user should not access the contents of the frame as the underlying memory may be deleted without warning.
-
readchannel
¶ Video input DMA channel
Type: AxiVDMA.S2MMChannel
-
writechannel
¶ Video output DMA channel
Type: AxiVDMA.MM2SChannel
-
class
MM2SChannel
(parent, interrupt)[source]¶ Bases:
object
DMA channel from memory to a video output.
Will continually repeat the most recent frame written.
-
cacheable_frames
¶ Whether frames should be stored in cacheable or non-cacheable memory
Type: bool
-
activeframe
¶
-
desiredframe
¶
-
framedelay
¶
-
mode
The video mode of the DMA, must be called prior to starting. If changed while the DMA channel is running the channel will be stopped
-
newframe
()[source]¶ Returns a frame of the appropriate size for the video mode.
The contents of the frame are undefined and should not be assumed to be black
Returns: Return type: numpy.ndarray video frame
-
parked
¶ Is the channel parked or running in circular buffer mode
-
reload
()[source]¶ Reload the configuration of the DMA. Should only be called by the _FrameList class or if you really know what you are doing
-
running
¶
-
setframe
(frame)[source]¶ Sets a frame without blocking or taking ownership. In most circumstances writeframe() is more appropriate
-
start
()[source]¶ Start the DMA channel with a blank screen. The mode must be set prior to calling or a RuntimeError will result.
-
-
class
S2MMChannel
(parent, interrupt)[source]¶ Bases:
object
Read channel of the Video DMA
Brings frames from the video input into memory. Hands ownership of the read frames to the user code.
-
cacheable_frames
¶ Whether frames should be stored in cacheable or non-cacheable memory
Type: bool
-
activeframe
¶ The frame index currently being processed by the DMA
This process requires clearing any error bits in the DMA channel
-
desiredframe
¶ The next frame index to the processed by the DMA
-
irqframecount
¶
-
mode
The video mode of the DMA. Must be set prior to starting. Changing this while the DMA is running will result in the DMA being stopped.
-
parked
¶ Is the channel parked or running in circular buffer mode
-
readframe
()[source]¶ Read a frame from the channel and return to the user
This function may block until a complete frame has been read. A single frame buffer is kept so the first frame read after a long pause in reading may return a stale frame. To ensure an up-to-date frame when starting processing video read an additional time before starting the processing loop.
Returns: Return type: numpy.ndarray of the video frame
-
readframe_async
()[source]¶ Read a frame from the channel, yielding instead of blocking if no data is available. See readframe for more details
-
reload
()[source]¶ Reload the configuration of the DMA. Should only be called by the _FrameList class or if you really know what you are doing
-
running
¶ Is the DMA channel running
-
tie
(channel)[source]¶ Ties an output channel to this input channel. This is used to pass video from input to output without invoking the CPU for each frame. Main use case is when some slow processing is being done on a subset of frames while the video is passed through directly to the output. Only one output may be tied to an output. The tie is broken either by calling tie(None) or writing a frame to the tied output channel.
-
-
bindto
= ['xilinx.com:ip:axi_vdma:6.2', 'xilinx.com:ip:axi_vdma:6.3']¶
-
pynq.lib.video.drm Module¶
-
class
pynq.lib.video.drm.
DrmDriver
(device_path, event_loop=None)[source]¶ Bases:
object
Driver for DRM-based output through the Linux kernel
This driver provides a zero-copy interface to the DRM subsystem exposing a similar API to the HDMI interface. The API should be configured with a PixelFormat containing a FourCC which will be passed to the Linux video subsystem.
Once configured frames can be retrieved using newframe which returns a numpy array mapped to a frame buffer. The frame can be switched using writeframe. Once a frame has been written it should not be modified as ownership has been transferred to the kernel.
-
configure
(mode, pixelformat)[source]¶ Configure the display output
Raises an exception if the initialisation fails.
Parameters: - mode (VideoMode) – The resolution to set the output display to
- pixelformat (PixelFormat) – The pixel format to use - must contain a fourcc
-
newframe
()[source]¶ Return a new frame which can later be written
Frames are not zeroed before being returned so the calling application should make sure the frame is fully written.
Returns: pynq.PynqBuffer Return type: numpy.ndarray mapped to a hardware frame
-
pynq.lib.video.dvi Module¶
-
class
pynq.lib.video.dvi.
HDMIInFrontend
(description)[source]¶ Bases:
pynq.lib.video.frontend.VideoInFrontend
,pynq.overlay.DefaultHierarchy
Class for interacting the with HDMI input frontend
This class is used for enabling the HDMI input and retrieving the mode of the incoming video
-
static
checkhierarchy
(description)[source]¶ Function to check if the driver matches a particular hierarchy
This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.
-
mode
-
static
-
class
pynq.lib.video.dvi.
HDMIOutFrontend
(description)[source]¶ Bases:
pynq.lib.video.frontend.VideoOutFrontend
,pynq.overlay.DefaultHierarchy
Class for interacting the HDMI output frontend
This class is used for enabling the HDMI output and setting the desired mode of the video stream
-
static
checkhierarchy
(description)[source]¶ Function to check if the driver matches a particular hierarchy
This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.
-
mode
Get or set the video mode for the HDMI output, must be set to one of the following resolutions:
640x480 800x600 1280x720 1280x1024 1920x1080
Any other resolution will result in a ValueError being raised. The bits per pixel will always be 24 when retrieved and ignored when set.
-
start
= None¶ Start the HDMI output - requires the that mode is already set
-
stop
= None¶ Stop the HDMI output
-
static
pynq.lib.video.frontend Module¶
pynq.lib.video.hierarchies Module¶
-
class
pynq.lib.video.hierarchies.
HDMIWrapper
(description)[source]¶ Bases:
pynq.overlay.DefaultHierarchy
Hierarchy driver for the entire video subsystem.
Exposes the input, output and video DMA as attributes. For most use cases the wrappers for the input and output pipelines are sufficient and the VDMA will not need to be used directly.
-
hdmi_in
¶ The HDMI input pipeline
Type: pynq.lib.video.HDMIIn
-
hdmi_out
¶ The HDMI output pipeline
Type: pynq.lib.video.HDMIOut
-
axi_vdma
¶ The video DMA.
Type: pynq.lib.video.AxiVDMA
-
static
checkhierarchy
(description)[source]¶ Function to check if the driver matches a particular hierarchy
This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.
-
-
class
pynq.lib.video.hierarchies.
VideoIn
(description, vdma=None)[source]¶ Bases:
pynq.overlay.DefaultHierarchy
Wrapper for the input video pipeline.
This wrapper assumes the following pipeline structure and naming
color_convert_in -> pixel_pack ->axi_vdma with vtc_in and axi_gpio_hdmiiin helper IP
-
frontend
¶ The HDMI frontend for signal detection
Type: pynq.lib.video.HDMIInFrontend
-
color_convert
¶ The input color format converter
Type: pynq.lib.video.ColorConverter
-
pixel_pack
¶ Converts the input pixel size to that required by the VDMA
Type: pynq.lib.video.PixelPacker
-
cacheable_frames
¶ Whether frames should be cacheable or non-cacheable
Only valid if a VDMA has been specified
-
static
checkhierarchy
(description)[source]¶ Function to check if the driver matches a particular hierarchy
This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.
-
colorspace
¶ The colorspace of the pipeline, can be changed without stopping the pipeline
-
configure
(pixelformat=<pynq.lib.video.common.PixelFormat object>)[source]¶ Configure the pipeline to use the specified pixel format.
If the pipeline is running it is stopped prior to the configuration being changed
Parameters: pixelformat (PixelFormat) – The pixel format to configure the pipeline for
-
mode
¶ Video mode of the input
-
-
class
pynq.lib.video.hierarchies.
VideoOut
(description, vdma=None)[source]¶ Bases:
pynq.overlay.DefaultHierarchy
Wrapper for the output video pipeline.
This wrapper assumes the following pipeline structure and naming
axi_vdma -> pixel_unpack -> color_convert -> frontend with vtc_out and axi_dynclk helper IP
-
frontend
¶ The HDMI frontend for mode setting
Type: pynq.lib.video.HDMIOutFrontend
-
color_convert
¶ The output color format converter
Type: pynq.lib.video.ColorConverter
-
pixel_unpack
¶ Converts the input pixel size to 24 bits-per-pixel
Type: pynq.lib.video.PixelPacker
-
cacheable_frames
¶ Whether frames should be cacheable or non-cacheable
Only valid if a VDMA has been specified
-
static
checkhierarchy
(description)[source]¶ Function to check if the driver matches a particular hierarchy
This function should be redefined in derived classes to return True if the description matches what is expected by the driver. The default implementation always returns False so that drivers that forget don’t get loaded for hierarchies they don’t expect.
-
colorspace
¶ Set the colorspace for the pipeline - can be done without stopping the pipeline
-
configure
(mode, pixelformat=None)[source]¶ Configure the pipeline to use the specified pixel format and size.
If the pipeline is running it is stopped prior to the configuration being changed
Parameters: - mode (VideoMode) – The video mode to output
- pixelformat (PixelFormat) – The pixel format to configure the pipeline for
-
mode
¶ The currently configured video mode
-
pynq.lib.video.pipeline Module¶
-
class
pynq.lib.video.pipeline.
ColorConverter
(description)[source]¶ Bases:
pynq.overlay.DefaultIP
Driver for the color space converter
The colorspace convert implements a 3x4 matrix for performing arbitrary linear color conversions. Each coefficient is represented as a 10 bit signed fixed point number with 2 integer bits. The result of the computation can visualised as a table
# in1 in2 in3 1 # out1 c1 c2 c3 c10 # out2 c4 c5 c6 c11 # out3 c7 c8 c9 c12
The color can be changed mid-stream.
-
colorspace
¶ The coefficients of the colorspace conversion
Type: list of float
-
bindto
= ['xilinx.com:hls:color_convert:1.0', 'xilinx.com:hls:color_convert_2:1.0']¶
-
colorspace
The colorspace to convert. See the class description for details of the coefficients. The coefficients are a list of floats of length 12
-
-
class
pynq.lib.video.pipeline.
PixelPacker
(description)[source]¶ Bases:
pynq.overlay.DefaultIP
Driver for the pixel format convert
Changes the number of bits per pixel in the video stream. The stream should be paused prior to the width being changed. This can be targeted at either a pixel_pack or a pixel_unpack IP core.For a packer the input is always 24 bits per pixel while for an unpacker the output 24 bits per pixel.
-
bindto
= ['xilinx.com:hls:pixel_pack:1.0', 'xilinx.com:hls:pixel_unpack:1.0', 'xilinx.com:hls:pixel_pack_2:1.0', 'xilinx.com:hls:pixel_unpack_2:1.0']¶
-
bits_per_pixel
¶ Number of bits per pixel in the stream
Valid values are 8, 24 and 32. The following table describes the operation for packing and unpacking for each width
Mode Pack Unpack 8 bpp Keep only the first channel Pad other channels with 0 16 bpp Dependent on resample Dependent on resample 24 bpp No change No change 32 bpp Pad channel 4 with 0 Discard channel 4
-
resample
¶ Perform chroma resampling in 16 bpp mode
Boolean property that only affects 16 bpp mode. If True then the two chroma channels are multiplexed on to the second output pixel, otherwise only the first and second channels are transferred and the third is discarded
-
pynq.lib.video.xilinx_hdmi Module¶
-
class
pynq.lib.video.xilinx_hdmi.
HdmiRxSs
(description)[source]¶ Bases:
pynq.overlay.DefaultIP
,pynq.lib.video.frontend.VideoInFrontend
Driver for the HDMI receiver subsystem
-
bindto
= ['xilinx.com:ip:v_hdmi_rx_ss:3.1']¶
-
load_edid
(data)[source]¶ Configure the EDID data exposed by the receiver
The EDID should be between 128 and 256 bytes depending on the resolutions desired. In order to trigger the EDID to be read by the source the HPD line should be toggled after the EDID has been loaded.
Parameters: data (bytes-like) – EDID data to load
-
mode
¶ Return the mode of the attached device
-
set_hpd
(value)[source]¶ Set the Host presence detect line
1 or True advertises the presence of a monitor to the source 0 or False shows a disconnected cable
Parameters: value (int or Boolean) – The desired value of the HPD line
-
-
class
pynq.lib.video.xilinx_hdmi.
HdmiTxSs
(description)[source]¶ Bases:
pynq.overlay.DefaultIP
,pynq.lib.video.frontend.VideoOutFrontend
Driver for the HDMI transmit subsystem
-
bindto
= ['xilinx.com:ip:v_hdmi_tx_ss:3.1']¶
-
read_edid
()[source]¶ Return the EDID of the attached monitor
Returns: bytes Return type: 256 bytes of EDID data
-
set_phy
(phy)[source]¶ Set the attached PHY
The subsystem must be attached to a Xilinx HDMI PHY to operate
Parameters: phy (HdmiVPhy) – The driver for the PHY
-