Best Practices for specifying Python package versions in AI notebooks
Q: Where can I find the list of Python dependencies?
A: For a list of current Python dependencies for developers creating Jupyter notebooks in AI Workbench, see https://supportdocs.blueconic.com/python/changelog/100-dependencies-changelog.html.
Q: Where can I find the AI Workbench API reference?
A: Review the BlueConic Python API documentation site for reference documentation. (This site includes a Changelog section that lists any change or upgrade to the Python package—e.g., the Python version, list of version dependencies.)
Q: Can developers use external libraries in AI Workbench?
A: Yes, you can install external libraries using pip. Libraries cannot be installed in AI Workbench if they are incompatible with Python 3, or if they require a specific native library to be installed.
Q: Why should we avoid recommending >= in version constraints?
A: Using >= for specifying version constraints can lead to compatibility issues, especially when major updates occur that introduce breaking changes. For example, if a package like Pandas releases a backwards-incompatible 3.0 version, using >= will automatically upgrade notebooks to that version. This could cause notebooks to break when the new version is incompatible with the existing code.
Q: Does the command pip install ipytest>=0.13.3 python-dateutil==2.8.2 markdown==3.4.1 work?
A: No, this command does not work as expected because >= needs to be quoted in Jupyter notebooks. Without quoting, the shell interprets >= as a redirection operator, leading to errors. To fix this, quote the version specification as shown here:
%pip install "ipytest>=0.13.3" python-dateutil==2.8.2 markdown==3.4.1
Q: What’s the best way to ensure backward-compatible upgrades?
A: If you want to allow users to upgrade to newer versions that maintain backward compatibility, you should use the ~= operator. This ensures that only patch versions can be updated, avoiding major version upgrades that might break the notebooks.
For example:
%pip install pandas~=2.0 ipytest~=0.13.3
This will allow any version updates within the 2.x range but will prevent the notebooks from automatically upgrading to Pandas 3.0 when it’s released.
This will also prevent an update to 2.1 or 2.2, you can use %pip install "pandas==2.*" to install any version in the 2.x range.
Q: Why is it important to manage version constraints carefully?
A: Package updates, especially major versions, can introduce breaking changes that render existing code incompatible. By using precise version constraints (== or ~=), you ensure that notebooks continue to run as expected and prevent unforeseen errors when newer versions of libraries are released or when a package used in the notebook requires a specific (maybe older) version of another package - for example:
%pip install pandas==1.5.1 'numpy<2.0'
Q: How should I handle an upcoming major version like Pandas 3.0?
A: To avoid future breakages from major version upgrades such as Pandas 3.0, it’s better to pin the version to the latest compatible one or allow upgrades only within the current major version (e.g., Pandas 2.x). For example:
%pip install pandas==2.2.3 ipytest==0.13.3
Or:
%pip install "pandas<=2.2.3" ipytest==0.13.3
Or:
You can also use `%pip install "pandas==2.*"`, which would also work in the hypothetical scenario if a 2.10 version were released.
This ensures that you can continue to receive bug fixes and minor upgrades within the 2.x series, while preventing migration to the 3.0 version until you are ready to test and adapt your code.
Q: How do I update the external libraries called in my AI Workbench notebooks?
A: Update the package versions used in your code to meet the version requirements in the Python dependencies page and Save your changes. For example, you would change
%pip install -q matplotlib==3.5.1 numpy==1.22.1 pyarrow==15.0.0 pytz==2021.3 ipytest==0.13.1 python-dateutil==2.8.2to
%pip install -q matplotlib==3.5.1 numpy==1.26.4 pyarrow==15.0.0 pytz==2021.3 ipytest==0.13.3Q: Who can I contact for questions or help resolving library/package updates?
Contact BlueConic Support if you have questions. Note that BlueConic Support cannot edit your AI notebooks but can help you understand if there are errors and how they should be resolved.