PYNQ Utils Module¶
The PYNQ utils module includes helper functions for installing and testing packages that use PYNQ.
Downloading Overlays with Setuptools¶
To avoid needing to put large bitstreams in source code repositories or on PyPI
PYNQ supports the use of link files. A link file is a file with the extension
.link
and contains a JSON dictionary of shell or board name matched against
the URL where the overlay can be downloaded, and the MD5 checksum of the file
{
"xilinx_u200_xdma_201830_2": {
"url": "https://link.to/u200.xclbin",
"md5sum": "da1e100gh8e7becb810976e37875de38"
}
"xilinx_u250_xdma_201830_2": {
"url": "https://link.to/u250.xclbin",
"md5sum": "1df38cf582c4c5d0c8e3ca38be8f1eb3"
}
}
In case the resource to be downloaded is global the url and md5sum entries should be at the top-level of the JSON dictionary. In this case, no device-based resolution will be performed.
{
"url": "https://link.to/resource.extension",
"md5sum": "1df38cf582c4c5d0c8e3ca38be8f1eb3"
}
The link files also supports compressed files in the url key. When a compressed file is provided, the PYNQ utils module will detect the extension and unpack the file automatically. The supported compressed files are shutil.get_unpack_formats().
PYNQ provides a download_overlays
setuptools command which will process any
link files in your repository and download overlays and resources for your
board and place them alongside the link files. To run the the download command
automatically at build time the utils module provides a build_py
command
class that can be used in place of the normal build_py
phase. For more
details on how to include this command in your setup.py see the
Python Packaging section for an example. Refer also to the
official Python documentation
and the setuptools documentation for more info on extending and reusing setuptools.
To download the overlays from your own setuptools pass the same functionality
is exposed through the download_overlays
function. For more information see
the module documentation available at pynq.utils Module.
Installing Notebooks Programmatically¶
The utils module contains the implementation behind the pynq get-notebooks
CLI command. Most commonly this should be called from the command line but the
functionality is also available through the deliver_notebooks
function. In
delivering the notebooks any link files in the source directory will be
evaluated if necessary. For the full details on the function and its arguments
again refer to the pynq.utils Module documentation.
Testing Notebooks¶
We have exposed infrastructure for testings notebooks that can be used as part
of an automated test setup. The run_notebook
function launches an instance
of Jupyter, executes all of the cells in the notebook and then returns an
object containing the results of the cells. The Python objects for each cell
can by retrieved using the _*
notation - the same as the IPython
environment. Note that objects which can’t be serialized are returned as a
string containing the result of the repr
function.
Notebooks are run in isolated environments so that any files created do not
pollute the package. Prior to starting Jupyter the entirety of the
notebook_path
directory is copied to a temporary location with the notebook
executed from there. For this reason the notebook name should always be given
as a relative path to the notebook_path
.
If you wish to inspect the output of the cells directly - e.g. to ensure that
rich display for an object is working correctly - the result object as an
outputs
property which contains the raw output from each cell in the
notebook format.
The following is a simple example of using the run_notebook
function as
part of a suite of pytests.
from pynq.utils import run_notebook
from os import path
NOTEBOOK_PATH = path.join(path.dirname(__file__), 'notebooks')
def test_notebook_1():
result = run_notebook('notebook_1.ipynb', NOTEBOOK_PATH)
assert result._3 == True # Cell three checks array equality
assert result._5 == 42