[Python] Basic dependency management with pyproject and uv (#6)

Sets us up to do a uv sync to set up the current Python environment with the correct dependencies. Note that this is just a facilitating mechanism for preparing the python environment; users are still free to set up the environment via any means they like.

By default the only dependency installed is the Python bindings (from llvm/eudsl) -- that is, enabling import mlir in Python. This is the common dependency for generating and running pipelines and for ingress. Optionally one can pass (a variant of) --extra ingress-torch to uv sync to also get the torch and torch-mlir dependencies -- that is, enabling import torch and import torch_mlir in Python. Per the included docs, it is recommended to first set up a fresh virtualenv, e.g. via uv venv.

The pyproject.toml comes with uv-specific sections so as to enable specifying from which "indices", i.e. Python package repositories, each specific package is expected to come from. This file pins particular nightlies/versions from these package repositories.

Author: rolfmorel

README.md

@@ -49,3 +49,22 @@ The planned work is:
 * **Schedule**: MLIR schedules to combine into pipelines and combination strategy on branch [TBD].
 * **Runtime**: Dependencies, tools and environments to run on various hardware on branch [TBD].
 * **Build System**: CMake / Bazel magic to check dependencies, track repositories, run CI on branch [TBD].
+
+## Getting up and running
+
+For the time being, `lighthouse` depends on just the Python bindings for [`mlir`](https://github.com/llvm/eudsl/releases).
+To install this dependency, obtain the [`uv`](https://docs.astral.sh/uv/getting-started/installation/#pypi) Python package manager and run the following in the root of the project:
+```
+$ uv venv  # Create a .venv virtualenv
+$ uv sync  # Install the `mlir-python-bindings` dependency into the virtualenv
+$ uv sync --extra ingress-torch-cpu  # Optionally install the dependencies for torch ingress
+```
+
+<details>
+<summary>
+A note on vendor-specific `torch` versions.
+</summary>
+For vendor-specific versions of `torch` use the targets `ingress-torch-nvidia`, `ingress-torch-rocm` or `ingress-torch-xpu` for Nvidia, AMD, and Intel-enabled versions, respectively.
+</details>
+
+To run the Python programs in this repo, either enter the virtual environment (`$ source .venv/bin/activate`) and execute a program _or_ execute each of the programs through `uv` (i.e. `$ uv run $EXE`), which will automatically run them inside the virtualenv.

ingress/Torch-MLIR/install-virtualenv.sh

@@ -1,7 +1,5 @@
 #!/usr/bin/env bash
 
-source mlir-gen-venv/bin/activate
-
 LAYERS=1024,2048,4096,512
 
 mkdir -p cache

ingress/mlir-gen/generate-linalg-3layer-mlp.sh

@@ -0,0 +1,64 @@
+[project]
+name = "lighthouse"
+version = "0.1.0a1"
+requires-python = ">=3.10,<3.13"  # Bounds are due to torch-mlir's packaging
+dependencies = [
+    "mlir-python-bindings==20251011+3af43e303",
+]
+
+[project.optional-dependencies]
+# Additional "targets" which pull in optional dependencies -- use `uv sync --extra TARGET`
+ingress_torch_cpu = [
+    "torch==v2.8.0+cpu",
+    "torch-mlir==20251003.589"
+]
+ingress_torch_nvidia = [
+    "torch==2.8.0",  # Nvidia-enabled version of torch
+    "torch-mlir==20251003.589"
+]
+ingress_torch_rocm = [
+    "torch==2.8.0+rocm6.4",  # AMD-enabled version of torch
+    "torch-mlir==20251003.589",
+    "pytorch_triton_rocm"  # Transitive dependency listed explicitly so that we can state which package repository it is supposed to come from
+]
+ingress_torch_xpu = [
+    "torch==2.8.0+xpu",  # Intel-enabled version of torch
+    "torch-mlir==20251003.589",
+    "pytorch_triton_xpu"  # Transitive dependency listed explicitly so that we can state which package repository it is supposed to come from
+]
+
+[tool.uv]
+# Declare that the following "targets" are mutually exclusive of one another
+conflicts = [
+    [
+        {extra = "ingress_torch_cpu" },
+        {extra = "ingress_torch_nvidia" },
+        {extra = "ingress_torch_rocm" },
+        {extra = "ingress_torch_xpu" }
+    ]
+]
+
+[tool.uv.sources]
+# Bind packages to particular package repositories
+mlir_python_bindings = { index = "eudsl" }
+torch = { index = "pytorch" }
+pytorch_triton_xpu = { index = "pytorch" }
+pytorch_triton_rocm = { index = "pytorch" }
+torch_mlir = { index = "torch_mlir" }
+
+# The following are the different non-Pypi package repositories we depend on
+[[tool.uv.index]]
+name = "eudsl"
+url = "https://llvm.github.io/eudsl"
+explicit = true
+format = "flat"
+
+[[tool.uv.index]]
+name = "pytorch"
+url = "https://download.pytorch.org/whl"
+explicit = true
+
+[[tool.uv.index]]
+name = "torch_mlir"
+url = "https://github.com/llvm/torch-mlir-release/releases/expanded_assets/dev-wheels"
+explicit = true