Can others reproduce your results?
Using uv and pixi in Python data analysis projects

Author

James Thomas
j.thomas@bristol.ac.uk

Reproducible programming environments are crucial for ensuring consistent and reliable software development. They enable developers to recreate exact configurations, facilitating collaboration and long-term maintenance of projects.

These talks demonstrate uv and pixi - environment management tools for Python, for PyPI and conda packages (respectively). They show how as a data scientist, research software engineer or scientific coder, you might take practical steps to incorporate them into your project.

Viewing the resources

Choose the tool you’d like to know more about:

uv

Play the video of the talk

Slides (with embedded videos)

Summary

  • uv automates the creation of virtual environments and the installation of specific versions of Python and PyPI (pip) packages inside them.
  • It manages a lockfile for you, that ensures the exact same versions of all your dependencies can be installed on a fresh system. This is especially useful if you’re collaborating with others. Unlike pip, it ensures all your dependencies are compatible with each other.
  • It helps you follow best-practices, such as creating projects with an importable package (for shared library functions), and defining project scripts.
  • It allows you to create isolated scripts which define their dependencies in a special comment at the top of the file.
  • It allows you to use temporarily add additional packages to environment (such as JupyterLab) to a disposable environment.
  • It allows you to use and install system-wide tools (such as the formatter, Ruff).

Files to version control

pyproject.toml
project settings
uv.lock
lockfile (do not edit by hand)

Key commands

First, install uv, restart your terminal, and run:

uv init
create a project
uv init --package
create a project with an importable package (for shared library functions)
uv init --script my_isolated_script.py
create a script that will run in an isolated environment
uv add pandas
install packages
uv run my_script.py
run a Python script in the environment (typing python is not required)
uv run jupyter lab
run an already-installed program in the environment (such as JupyterLab)
uv run --with jupyter jupyter lab
temporarily add a package (that is not installed) to an environment and run a command from it (such as JupyterLab)
uv run my_task
run a project script (defined in pyproject.toml)
uv tree
see installed packages and dependencies
uv tool install ruff
install a tool that can be used system-wide (such as the formatter, Ruff)
uvx ruff
similar to above, but don’t formally install the tool, just download and run it

Example data science project layout

Further reading

pixi

Play the video of the talk

Slides (with embedded videos)

Summary

  • pixi automates the creation of conda environments and the installation of specific versions of Python, conda and PyPI (pip) packages inside them.
  • It manages a lockfile for you, that ensures the exact same versions of all your dependencies can be installed on a fresh system. This is especially useful if you’re collaborating with others. Unlike conda, it can work with collaborators using different platforms. Unlike pip, it ensures all your dependencies are compatible with each other.
  • It helps you follow best-practices, such as creating projects (or workspaces) with an importable package (for shared library functions), and defining project tasks.
  • It allows you to use and install system-wide tools (such as the formatter, Ruff).

Files to version control

pixi.toml/pyproject.toml
project (or workspace) settings
pixi.lock
lockfile (do not edit by hand)

Key commands

First, install pixi, restart your terminal, and run:

pixi init
create a project (or workspace), using pixi.toml as the project configuration file.
pixi init --format pyproject
create a project with an importable package (for shared library functions), using pyproject.toml as the project configuration file.
pixi add pandas
install packages
pixi run python my_script.py
run a Python script in the environment
pixi run jupyter lab
run an installed program in the environment (such as JupyterLab)
pixi run my_task
run a project task (defined in pixi.toml/pyproject.toml)
pixi tree
see installed packages and dependencies
pixi workspace platform add linux-64 osx-arm64 win-64
configure the project to support multiple platforms that other developers may be using
pixi global install ruff
install a tool that can be used system-wide (such as the formatter, Ruff)
pixi exec ruff
similar to above, but don’t formally install the tool, just download and run it

Example data science project layout

Further reading

Development

For information on changing these slides or running the live demos, see the GitHub repo.

Licence

CC BY 4.0