Migrate dependency management to pyproject.toml with uv

[matrixise]

Description

Complete migration from the legacy pip-tools workflow using requirements/*.in files to a modern Python packaging approach using pyproject.toml with uv for dependency management.

This migration consolidates all dependency declarations into a single source of truth (pyproject.toml) following PEP 621 (project metadata) and PEP 735 (dependency groups) standards.

Type of Change

• Refactoring
• Documentation update
• Bug fix
• New feature

Key Changes

Structure

• Replaced requirements/main.in, requirements/dev.in, requirements/production.in → pyproject.toml
• Generated uv.lock as universal lock file (102 packages resolved)
• Auto-generate requirements.txt for Heroku deployment compatibility
• Centralized all tool configurations (ruff, coverage, isort) in pyproject.toml
• Using Hatchling as build backend

New Workflow

# Update dependencies
uv lock

# Install dependencies (development)
uv sync

# Install dependencies (production)
uv sync --no-dev --group production

# Export for Heroku
task dependencies:export

# Upgrade specific package
task dependencies:upgrade:package PACKAGE=django

Files Modified

• pyproject.toml - New single source of truth
• uv.lock - Universal lock file
• Taskfile.yaml - Updated dependency management tasks
• toast.yml - Updated Docker-based dependency tasks
• Dockerfile - Use uv sync instead of pip install
• .github/workflows/test.yml - Updated CI to use uv
• CLAUDE.md, README.md, DEVELOPMENT.md, CONTRIBUTING.md - Updated documentation

Files Removed

• requirements/main.in, requirements/dev.in, requirements/production.in
• requirements/main.txt, requirements/dev.txt, requirements/production.txt
• requirements/ directory (now empty)
• pythonie/setup.py (replaced by pyproject.toml)

Benefits

1. Single source of truth - All dependencies in pyproject.toml
2. Faster resolution - uv is 10-100x faster than pip
3. Modern standards - PEP 621 + PEP 735 (approved Oct 2024)
4. Centralized config - All tool settings in one file
5. Better reproducibility - Universal lock file with hashes
6. Simplified commands - One command to install instead of three

How to Test

Local Setup

# Clean environment
rm -rf .venv pythonie-venv

# Install with new system
uv sync

# Run tests
python pythonie/manage.py test pythonie --settings=pythonie.settings.tests --verbosity=2

Docker Setup

# Rebuild Docker image
task docker:build

# Run tests in Docker
task tests

Expected Results

• ✅ All dependencies installed successfully
• ✅ All 33 tests pass
• ✅ requirements.txt generated with correct header
• ✅ Development workflow unchanged from user perspective

Testing Performed

• ✅ Clean installation with `uv sync`
• ✅ All 33 tests pass (ran in 0.156s)
• ✅ Generated `requirements.txt` for Heroku with auto-generated header
• ✅ Verified all dependencies resolved correctly (102 packages)

Checklist

• Code follows project style guidelines
• Tests pass locally (`uv sync` + full test suite)
• New tests added (N/A - infrastructure change)
• Documentation updated (CLAUDE.md, README.md, DEVELOPMENT.md, CONTRIBUTING.md)
• Backward compatibility maintained (requirements.txt still generated for Heroku)
• All old dependency files removed
• uv.lock committed to version control

Migration Impact

Developers

• Must run `uv sync` instead of `pip install -r requirements.txt`
• New commands via Taskfile (but similar to before)

CI/CD

• GitHub Actions updated to use `uv sync`
• All tests passing with new setup

Deployment (Heroku)

• No impact - still uses `requirements.txt` (auto-generated)
• Deployment process unchanged

References

• Fixes 🔧 Migrate from pip to uv for dependency management #171
• PEP 621: Storing project metadata in pyproject.toml
• PEP 735: Dependency Groups in pyproject.toml (Oct 2024)
• uv documentation

---

Note: This is a significant infrastructure change, but maintains full backward compatibility for deployment while modernizing the development workflow.