Python Packaging

Note: This repo is based on the book Publishing Python Packages: Test, Share, and Automate Your Projects. The code examples from the book are available here: GitHub.

Package Features

Important: This package uses Cython extensions that compile to platform-specific binaries. We use cibuildwheel in CI/CD to build wheels for Linux, Windows, and macOS (Intel + Apple Silicon). Without pre-built wheels, users would need a C compiler and build tools installed.

Alternative build backends:

Why Maturin for Rust?

Maturin is specifically designed for building Python packages with Rust extensions using PyO3 bindings:

Key differences from Cython:

How Rust bindings work:

  1. Write performance-critical code in Rust
  2. Use PyO3 to create Python bindings (exposes Rust functions to Python)
  3. Maturin handles compilation and wheel building automatically
  4. Result: Fast Rust code callable from Python

Advantages of Rust + Maturin:

When to use:

Project Configuration Files

setup.py

Handles the Cython extension build. Defines the harmonic_mean extension module with numpy includes and compiler directives for coverage tracing.

setup.cfg

Contains package metadata and configuration for all tools:

pyproject.toml

Specifies the build system requirements: setuptools, wheel, cython, and numpy needed to build the package.

MANIFEST.in

Controls which non-Python files are included in the source distribution:

Tip: Modern Python packages can use pyproject.toml for everything, but this project uses the traditional setup.cfg + setup.py approach, which is still widely used and well-supported.

Building the Package

To build the package, run the following command:

pyproject-build

To install the package locally, use:

python -m pip install -e .  # Editable install for development

After installation, the harmony command is available:

harmony  # Runs the harmonic mean calculator

Available Tox Commands

tox

tox is a tool for running tests in multiple environments.

Run tests on specific Python versions:

tox -e py39          # Test on Python 3.9
tox -e py310         # Test on Python 3.10

Run quality checks:

tox -e format        # Format code with black
tox -e lint          # Lint with flake8
tox -e typecheck     # Type check with mypy
tox -e isort         # Sort imports

Run everything in parallel:

tox -p -e py39,py310,typecheck,format,lint

CI/CD with GitHub Actions

github action logo

The workflow automatically builds wheels for Linux, Windows, and macOS (Intel + Apple Silicon) using cibuildwheel.

Configuration:

env:
  CIBW_ARCHS_MACOS: "x86_64 arm64"  # Separate wheels for Intel and Apple Silicon
  CIBW_BUILD: "cp39-* cp310-*"       # Python 3.9 and 3.10 only

Publishing to Test PyPI:

git tag v0.1.0
git push origin v0.1.0

Install from Test PyPI:

pip install --index-url https://test.pypi.org/simple/ first-python-package

Modern Python Tooling with uv

uv

uv is a next-generation Python package manager written in Rust, designed to replace multiple tools with a single, fast solution.

What is uv?

uv is an extremely fast Python package and project manager that aims to replace:

Key advantages:

Publishing with uv

Instead of using twine or GitHub Actions, you can publish directly with uv:

# Build the package
uv build

# Publish to PyPI
uv publish

# Publish to Test PyPI
uv publish --publish-url https://test.pypi.org/legacy/

Authentication:

# Set PyPI token
export UV_PUBLISH_TOKEN="pypi-..."

# Or use keyring
uv publish --keyring-provider subprocess

uv vs Traditional Workflow

Traditional (this project):

python -m build                    # Build package
python -m twine upload dist/*      # Upload to PyPI
pip install -r requirements.txt    # Install dependencies

With uv:

uv build                          # Build package
uv publish                        # Upload to PyPI
uv sync                           # Install dependencies (from lockfile)

Why uv is the Future

  1. Speed: Resolves and installs dependencies 10-100x faster than pip
  2. Reliability: Lockfiles ensure reproducible environments across machines
  3. Simplicity: One tool instead of pip, venv, pip-tools, twine, etc.
  4. Modern: Built with Rust, designed for today’s Python ecosystem
  5. Compatible: Works with existing pyproject.toml and requirements.txt

Migrating to uv

This project could be simplified with uv:

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create project
uv init

# Add dependencies
uv add numpy termcolor

# Add dev dependencies
uv add --dev pytest black flake8 mypy

# Run commands
uv run pytest
uv run black .

# Build and publish
uv build
uv publish

Tip: While this project uses traditional tools (setuptools, pip, twine), uv represents the future of Python packaging. It’s production-ready and actively maintained by Astral (creators of Ruff). Consider using uv for new projects!

Type Checking with mypy

mypy logo

mypy is a static type checker for Python that helps catch bugs before runtime:

tox -e typecheck  # Run mypy on this project

Configuration (in setup.cfg):

[mypy]
python_version = 3.10
warn_unused_configs = True
check_untyped_defs = True

Why use type checking?

Example from this project:

def harmonic_mean(numbers: list[float]) -> float:
    """Calculate harmonic mean with type hints."""
    return len(numbers) / sum(1/x for x in numbers)

mypy verifies that you pass a list of floats and get a float back, catching type mismatches at development time.

Tip: Modern Python projects should use type hints and mypy. They’re optional but highly recommended for maintainability and catching bugs early.

Notes

Other Tools

About .egg and .egg-info files

.egg-info directories (like src/first_python_package.egg-info/) are created during package installation and contain metadata about the installed package. They’re still used today.

.egg files were the predecessor to wheels (.whl):

Note: You may see .egg-info directories in your project - this is normal! They’re created by pip install -e . and contain package metadata. Modern Python uses wheels (.whl) for distribution, not .egg files.

Resources

License

MIT License - see LICENSE file for details.