SIGN IN SIGN UP

feat: Add a venv-based build environment provider (#14070)

By using real virtual environments, pip behaves more similarly to
python-build and with less buggy isolation. However, since the
venv-based environment does differ in numerous respects, we need to
introduce this feature in stages. It will be first be available with
`--use-feature=venv-isolation`

A few noteworthy details:

(1) Since the sitecustomize.py trick is gone, we need to call the venv's
  Python executable directly to pick up the right site-packages
  directories.

(2) We no longer prepend the environment's site to PYTHONPATH as all of
  the backend calls will be made with the venv's Python executable. If
  they spawn more Python subprocesses, the PATH adjustment will ensure
  that the venv's Python executable is still used. (This doesn't apply
  for system level Python *tools*, but those should be able to access
  their own site paths.)

(3) The context object that venv.ensure_directories returns containing
  all of the key paths we need to interact with the virtual environment
  was only documented in Python 3.12, leading to a ton of fallbacks
  to handle versions that don't provide these paths directly. It sucks,
  I'm sorry. See the PR for all of the glory details.

Some known behavioural differences:

(1) Backend calls are now able to use system Python tools from the
  system environment as isolation doesn't apply to such Python processes
  anymore (since the sitecustomize.py file was dropped). However, the
  backend isn't able to import external libraries, of course.

(2) There is no more site-packages layering, one for the static build
  backend dependencies, and one for the dynamic injected dependencies.
  They are all installed to the same location since nested virtual
  environments aren't really a thing. This means all dependencies
  requested by a backend must be resolvable altogether (which IMO
  seems fair).

(3) The venv-based build environment has limited compatibility with the
  inprocess build-dep installer as due to (2), we now need to be able to
  uninstall conflicting static dependencies while dynamic dependencies
  are installed. This doesn't work... since pip has no proper support
  for operating with another Python environment inprocess. This is
  fixable, but will take some time. Most builds should work just fine
  though, since in practice build dependencies usually don't conflict.

(4) The build environment's site-packages no longer overrides stdlib
  modules. This occured with the virtual build environment as it set
  PYTHONPATH which prepended said site directories to sys.path for
  backend subprocesses. This shouldn't break anyone, but who knows what
  can happen.

Regrettably, it's unlikely that we will be able to replace virtual build
environments wholesale with venvs as certain platforms do not support or
include the venv module (including Android, iOS, probably certain Linux
distributions... although not Debian-based distros!)

One last thing, I also fixed a pre-existing bug where we forgot to enter
the build environment context while installing the build system or
dynamic build dependencies. This was mostly "OK" with the legacy
isolation mechanism due to --ignore-installed but is problematic
for venv isolation as it doesn't use flag (causing pip to think the
build dependencies are already installed if available through an
ambient PYTHONPATH which the context normally patches away).
R
Richard Si committed
5c9ec602159d5d2b03fe32e63614925287b1b0d3
Parent: 131e63d
Committed by GitHub <noreply@github.com> on 7/7/2026, 11:56:11 PM