pynq.lib.video Module

The pynq.lib.video module is a driver capturing streaming HDMI input, producing streaming HDMI output and hardware-accelerated colorspace conversion.

class pynq.lib.video.AxiVDMA(description, framecount=4)

Bases: pynq.overlay.DefaultIP

Driver class for the Xilinx VideoDMA IP core

The driver is split into input and output channels are exposed using the readchannel and writechannel attributes. Each channel has start and stop methods to control the data transfer. All channels MUST be stopped before reprogramming the bitstream or inconsistent behaviour may result.

The DMA uses a single ownership model of frames in that frames are either owned by the DMA or the user code but not both. S2MMChannel.readframe and MM2SChannel.newframe both return a frame to the user. It is the user’s responsibility to either free the frame using the freebuffer() method or to hand ownership back to the DMA using MM2SChannel.writeframe. Once ownership has been returned the user should not access the contents of the frame as the underlying memory may be deleted without warning.

readchannel

AxiVDMA.S2MMChannel – Video input DMA channel

writechannel

AxiVDMA.MM2SChannel – Video output DMA channel

class MM2SChannel(parent, interrupt)

Bases: object

DMA channel from memory to a video output.

Will continually repeat the most recent frame written.

mode

VideoMode – Video mode of the DMA channel

activeframe

desiredframe

framedelay

mode

The video mode of the DMA, must be called prior to starting. If changed while the DMA channel is running the channel will be stopped

newframe()

Returns a frame of the appropriate size for the video mode.

The contents of the frame are undefined and should not be assumed to be black

Returns: Return type:numpy.ndarray video frame

parked

Is the channel parked or running in circular buffer mode

reload()

Reload the configuration of the DMA. Should only be called by the _FrameList class or if you really know what you are doing

reset()

Soft reset the DMA channel

running

setframe(frame)

Sets a frame without blocking or taking ownership. In most circumstances writeframe() is more appropriate

start()

Start the DMA channel with a blank screen. The mode must be set prior to calling or a RuntimeError will result.

stop()

Stop the DMA channel and empty the frame cache

writeframe(frame)

Schedule the specified frame to be the next one displayed. Assumes ownership of frame which should no longer be modified by the user. May block if there is already a frame scheduled.

writeframe_async(frame)

Same as writeframe() but yields instead of blocking if a frame is already scheduled

class S2MMChannel(parent, interrupt)

Bases: object

Read channel of the Video DMA

Brings frames from the video input into memory. Hands ownership of the read frames to the user code.

mode

VideoMode – The video mode of the DMA channel

activeframe

The frame index currently being processed by the DMA

This process requires clearing any error bits in the DMA channel

desiredframe

The next frame index to the processed by the DMA

irqframecount

mode

The video mode of the DMA. Must be set prior to starting. Changing this while the DMA is running will result in the DMA being stopped.

parked

Is the channel parked or running in circular buffer mode

readframe()

Read a frame from the channel and return to the user

This function may block until a complete frame has been read. A single frame buffer is kept so the first frame read after a long pause in reading may return a stale frame. To ensure an up-to-date frame when starting processing video read an additional time before starting the processing loop.

Returns: Return type:numpy.ndarray of the video frame

readframe_async()

Read a frame from the channel, yielding instead of blocking if no data is available. See readframe for more details

reload()

Reload the configuration of the DMA. Should only be called by the _FrameList class or if you really know what you are doing

reset()

Soft reset the DMA. Finishes all transfers before starting the reset process

running

Is the DMA channel running

start()

Start the DMA. The mode must be set prior to this being called

stop()

Stops the DMA, clears the frame cache and unhooks any tied outputs

tie(channel)

Ties an output channel to this input channel. This is used to pass video from input to output without invoking the CPU for each frame. Main use case is when some slow processing is being done on a subset of frames while the video is passed through directly to the output. Only one output may be tied to an output. The tie is broken either by calling tie(None) or writing a frame to the tied output channel.

bindto = ['xilinx.com:ip:axi_vdma:6.2', 'xilinx.com:ip:axi_vdma:6.3']

class pynq.lib.video.ColorConverter(description)

Bases: pynq.overlay.DefaultIP

Driver for the color space converter

The colorspace convert implements a 3x4 matrix for performing arbitrary linear color conversions. Each coefficient is represented as a 10 bit signed fixed point number with 2 integer bits. The result of the computation can visualised as a table

# in1 in2 in3 1 # out1 c1 c2 c3 c10 # out2 c4 c5 c6 c11 # out3 c7 c8 c9 c12

The color can be changed mid-stream.

colorspace

list of float – The coefficients of the colorspace conversion

bindto = ['xilinx.com:hls:color_convert:1.0']

colorspace

The colorspace to convert. See the class description for details of the coefficients. The coefficients are a list of floats of length 12

class pynq.lib.video.HDMIIn(description, vdma=None)

Bases: pynq.overlay.DefaultHierarchy

Wrapper for the input video pipeline of the Pynq-Z1 base overlay

This wrapper assumes the following pipeline structure and naming

color_convert_in -> pixel_pack ->axi_vdma with vtc_in and axi_gpio_hdmiiin helper IP

frontend

pynq.lib.video.HDMIInFrontend – The HDMI frontend for signal detection

color_convert

pynq.lib.video.ColorConverter – The input color format converter

pixel_pack

pynq.lib.video.PixelPacker – Converts the input pixel size to that required by the VDMA

static checkhierarchy(description)

close()

Uninitialise the drivers, stopping the pipeline beforehand

colorspace

The colorspace of the pipeline, can be changed without stopping the pipeline

configure(pixelformat=<pynq.lib.video.PixelFormat object>)

Configure the pipeline to use the specified pixel format.

If the pipeline is running it is stopped prior to the configuration being changed

Parameters:pixelformat (PixelFormat) – The pixel format to configure the pipeline for

mode

Video mode of the input

readframe()

Read a video frame

See AxiVDMA.S2MMChannel.readframe for details

readframe_async()

Read a video frame

See AxiVDMA.S2MMChannel.readframe for details

start()

Start the pipeline

stop()

Stop the pipeline

tie(output)

Mirror the video input on to an output channel

Parameters:output (HDMIOut) – The output to mirror on to

class pynq.lib.video.HDMIInFrontend(description)

Bases: pynq.overlay.DefaultHierarchy

Class for interacting the with HDMI input frontend

This class is used for enabling the HDMI input and retrieving the mode of the incoming video

mode

VideoMode – The detected mode of the incoming video stream

static checkhierarchy(description)

mode

start(init_timeout=60)

Method that blocks until the video mode is successfully detected

stop()

Currently empty function included for symmetry with the HDMIOutFrontend class

class pynq.lib.video.HDMIOut(description, vdma=None)

Bases: pynq.overlay.DefaultHierarchy

Wrapper for the output video pipeline of the Pynq-Z1 base overlay

This wrapper assumes the following pipeline structure and naming

axi_vdma -> pixel_unpack -> color_convert -> frontend with vtc_out and axi_dynclk helper IP

frontend

pynq.lib.video.HDMIOutFrontend – The HDMI frontend for mode setting

color_convert

pynq.lib.video.ColorConverter – The output color format converter

pixel_unpack

pynq.lib.video.PixelPacker – Converts the input pixel size to 24 bits-per-pixel

static checkhierarchy(description)

close()

Close the pipeline an unintialise the drivers

colorspace

Set the colorspace for the pipeline - can be done without stopping the pipeline

configure(mode, pixelformat=None)

Configure the pipeline to use the specified pixel format and size.

If the pipeline is running it is stopped prior to the configuration being changed

Parameters:

• mode (VideoMode) – The video mode to output
• pixelformat (PixelFormat) – The pixel format to configure the pipeline for

mode

The currently configured video mode

newframe()

Return an unintialised video frame of the correct type for the pipeline

start()

Start the pipeline

stop()

Stop the pipeline

writeframe(frame)

Write the frame to the video output

See AxiVDMA.MM2SChannel.writeframe for more details

writeframe_async(frame)

Write the frame to the video output

See AxiVDMA.MM2SChannel.writeframe for more details

class pynq.lib.video.HDMIOutFrontend(description)

Bases: pynq.overlay.DefaultHierarchy

Class for interacting the HDMI output frontend

This class is used for enabling the HDMI output and setting the desired mode of the video stream

mode

VideoMode – Desired mode for the output video. Must be set prior to calling start

static checkhierarchy(description)

mode

Get or set the video mode for the HDMI output, must be set to one of the following resolutions:

640x480 800x600 1280x720 1280x1024 1920x1080

Any other resolution will result in a ValueError being raised. The bits per pixel will always be 24 when retrieved and ignored when set.

start = None

Start the HDMI output - requires the that mode is already set

stop = None

Stop the HDMI output

class pynq.lib.video.HDMIWrapper(description)

Bases: pynq.overlay.DefaultHierarchy

Hierarchy driver for the entire Pynq-Z1 video subsystem.

Exposes the input, output and video DMA as attributes. For most use cases the wrappers for the input and output pipelines are sufficient and the VDMA will not need to be used directly.

hdmi_in

pynq.lib.video.HDMIIn – The HDMI input pipeline

hdmi_out

pynq.lib.video.HDMIOut – The HDMI output pipeline

axi_vdma

pynq.lib.video.AxiVDMA – The video DMA.

static checkhierarchy(description)

class pynq.lib.video.PixelFormat(bits_per_pixel, in_color, out_color)

Bases: object

Wrapper for all of the information about a video format

bits_per_pixel

int – Number of bits for each pixel

in_color

list of float – Coefficients from BGR stream to pixel format

out_color

list of float – Coefficient from pixel format to BGR stream

class pynq.lib.video.PixelPacker(description)

Bases: pynq.overlay.DefaultIP

Driver for the pixel format convert

Changes the number of bits per pixel in the video stream. The stream should be paused prior to the width being changed. This can be targeted at either a pixel_pack or a pixel_unpack IP core.For a packer the input is always 24 bits per pixel while for an unpacker the output 24 bits per pixel.

bindto = ['xilinx.com:hls:pixel_pack:1.0', 'xilinx.com:hls:pixel_unpack:1.0']

bits_per_pixel

Number of bits per pixel in the stream

Valid values are 8, 24 and 32. The following table describes the operation for packing and unpacking for each width

Mode Pack Unpack 8 bpp Keep only the first channel Pad other channels with 0 16 bpp Dependent on resample Dependent on resample 24 bpp No change No change 32 bpp Pad channel 4 with 0 Discard channel 4

resample

Perform chroma resampling in 16 bpp mode

Boolean property that only affects 16 bpp mode. If True then the two chroma channels are multiplexed on to the second output pixel, otherwise only the first and second channels are transferred and the third is discarded

class pynq.lib.video.VideoMode(width, height, bits_per_pixel, stride=None)

Bases: object

Class for holding the information about a video mode

height

int – Height of the video frame in lines

width

int – Width of the video frame in pixels

stride

int – Width of a line in the video frame in bytes

bits_per_pixel

int – Bits per pixel

bytes_per_Pixel

int – Bytes required to represent each pixel

shape

tuple of int – Numpy-style tuple describing the video frame