feat: Add a venv-based build environment provider (#14070)
By using real virtual environments, pip behaves more similarly to python-build and with less buggy isolation. However, since the venv-based environment does differ in numerous respects, we need to introduce this feature in stages. It will be first be available with `--use-feature=venv-isolation` A few noteworthy details: (1) Since the sitecustomize.py trick is gone, we need to call the venv's Python executable directly to pick up the right site-packages directories. (2) We no longer prepend the environment's site to PYTHONPATH as all of the backend calls will be made with the venv's Python executable. If they spawn more Python subprocesses, the PATH adjustment will ensure that the venv's Python executable is still used. (This doesn't apply for system level Python *tools*, but those should be able to access their own site paths.) (3) The context object that venv.ensure_directories returns containing all of the key paths we need to interact with the virtual environment was only documented in Python 3.12, leading to a ton of fallbacks to handle versions that don't provide these paths directly. It sucks, I'm sorry. See the PR for all of the glory details. Some known behavioural differences: (1) Backend calls are now able to use system Python tools from the system environment as isolation doesn't apply to such Python processes anymore (since the sitecustomize.py file was dropped). However, the backend isn't able to import external libraries, of course. (2) There is no more site-packages layering, one for the static build backend dependencies, and one for the dynamic injected dependencies. They are all installed to the same location since nested virtual environments aren't really a thing. This means all dependencies requested by a backend must be resolvable altogether (which IMO seems fair). (3) The venv-based build environment has limited compatibility with the inprocess build-dep installer as due to (2), we now need to be able to uninstall conflicting static dependencies while dynamic dependencies are installed. This doesn't work... since pip has no proper support for operating with another Python environment inprocess. This is fixable, but will take some time. Most builds should work just fine though, since in practice build dependencies usually don't conflict. (4) The build environment's site-packages no longer overrides stdlib modules. This occured with the virtual build environment as it set PYTHONPATH which prepended said site directories to sys.path for backend subprocesses. This shouldn't break anyone, but who knows what can happen. Regrettably, it's unlikely that we will be able to replace virtual build environments wholesale with venvs as certain platforms do not support or include the venv module (including Android, iOS, probably certain Linux distributions... although not Debian-based distros!) One last thing, I also fixed a pre-existing bug where we forgot to enter the build environment context while installing the build system or dynamic build dependencies. This was mostly "OK" with the legacy isolation mechanism due to --ignore-installed but is problematic for venv isolation as it doesn't use flag (causing pip to think the build dependencies are already installed if available through an ambient PYTHONPATH which the context normally patches away).
R
Richard Si committed
5c9ec602159d5d2b03fe32e63614925287b1b0d3
Parent: 131e63d
Committed by GitHub <noreply@github.com>
on 7/7/2026, 11:56:11 PM