Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error installing python packages using uv #2411

Open
AlejandroGomezFrieiro opened this issue Nov 15, 2024 · 2 comments · May be fixed by #2478
Open

Error installing python packages using uv #2411

AlejandroGomezFrieiro opened this issue Nov 15, 2024 · 2 comments · May be fixed by #2478
Labels
bug Something isn't working triage Issue needs triage

Comments

@AlejandroGomezFrieiro
Copy link

What happened?

When settings up devbox shell, setting up uv as a package to use as an installer, and then trying to use it to install numpy (or any other package) it complains with

error: Failed to install: numpy-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (numpy==2.1.3)
  Caused by: failed to create directory `/nix/store/m85llqyd79hjm4cbgx8hds2rarbslc1s-python3-3.11.2/lib/python3.11/site-packages/numpy`
  Caused by: Permission denied (os error 13)

Steps to reproduce

  1. Setup a python devbox with uv
  2. devbox shell
  3. uv pip install numpy

Command

No response

devbox.json

{
    "$schema": "https://raw.githubusercontent.com/jetify-com/devbox/0.13.6/.schema/devbox.schema.json",
      "packages": [
        "[email protected]",
        "uv@latest",
        "git@latest"
      ],

    "shell": {
    "init_hook": [
      "source $VENV_DIR/bin/activate",
    ],
      "scripts": {
        "test": [
          "echo \"Error: no test specified\" && exit 1"
        ]
      }
    }
  }

Devbox version

0.13.6

Nix version

nix (Nix) 2.24.7

What system does this bug occur on?

Linux (x86-64)

Debug logs

No response
@AlejandroGomezFrieiro AlejandroGomezFrieiro added bug Something isn't working triage Issue needs triage labels Nov 15, 2024
@yemaney
Copy link

yemaney commented Dec 16, 2024 •
edited
Loading

I've also had the same issue, and tried to look into it. Although my error looks slightly different. Ended up resolving it by updating the UV_PYTHON environment variable to point to the python under the .venv folder created by devbox.

Explanation

There are two locations where python executable is available in a devbox project.

  1. .devbox/nix/profile/default/bin/python: default UV_PYTHON
  2. .venv/bin/python: symlink to python under .devbox (above)

Each of them have a related site-packages directory, where installed libraries are saved to.

The UV_PYTHON default value will mean it will try to install and save libraries to the site-packages from (1), but this directory is not writable, probably because its contents are symlinked to /nix/store/. Making sure UV_PYTHON is set to the python under .venv will allow saving the installed libraries to the site-packages from (2) which is writable.

Related sources:

Error

$ uv pip install numpy
Using Python 3.12.1 environment at /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1
error: The interpreter at /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1 is externally managed, and indicates the following:

  This command has been disabled as it tries to modify the immutable
  `/nix/store` filesystem.

  To use Python with Nix and nixpkgs, have a look at the online documentation:
  <https://nixos.org/manual/nixpkgs/stable/

Consider creating a virtual environment with `uv venv`.

Fix

{
  "env": {
    "UV_PYTHON": "$PWD/.venv/bin/python"
  },
  "packages": [
    "python@latest",
    "uv@latest"
  ],
  "shell": {
    "init_hook": [
      ". $VENV_DIR/bin/activate",
      "uv pip install  -r requirements.txt"
    ],
    "scripts": {
      "run_test": "python main.py"
    }
  }
}
$ devbox shell
Starting a devbox shell...
Warning: Your devbox environment may be out of date. Run refresh-global to update it.
Virtual environment directory doesn't exist. Creating new one...
Resolved 5 packages in 25ms
Installed 5 packages in 17ms
 + emoji==2.1.0
 + iniconfig==2.0.0
 + packaging==24.2
 + pluggy==1.5.0
 + pytest==8.3.4
$ echo $UV_PYTHON
/home/ubuntu/python-uv/.venv/bin/python
$ uv pip install numpy
Resolved 1 package in 108ms
Installed 1 package in 101ms
 + numpy==2.2.0

Investigating

  • Regular python
$ echo $VENV_DIR
/home/ubuntu/python-uv/.venv

# follow path of python
$ which python
/home/ubuntu/python-uv/.venv/bin/python
$ ls -l /home/ubuntu/python-uv/.venv/bin/python
lrwxrwxrwx 1 ubuntu ubuntu 61 Dec 16 01:13 /home/ubuntu/python-uv/.venv/bin/python -> /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python
$ ls -l /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python
lrwxrwxrwx 1 ubuntu ubuntu 69 Jan  1  1970 /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python -> /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1/bin/python

# site-packages under .venv is writable
$ ls -l .venv/lib/python3.12/
total 4
drwxrwxr-x 15 ubuntu ubuntu 4096 Dec 16 02:15 site-packages
  • uv
$ echo $UV_PYTHON
/home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python
$ ls -l $UV_PYTHON
lrwxrwxrwx 1 ubuntu ubuntu 69 Jan  1  1970 /home/ubuntu/python-uv/.devbox/nix/profile/default/bin/python -> /nix/store/dnli4467fjzywkgwral3hr3xx33f8x2v-python3-3.12.1/bin/python

# site-packages under .devbox/ is not writeable
$ ls -l  .devbox/nix/profile/default/lib/

$ ls -l  .devbox/nix/profile/default/lib/python3.12/
dr-xr-xr-x 2 ubuntu ubuntu 4096 Dec 16 02:38 site-packages

@MarkValadez
Copy link

I ran into this issue, I found that "UV_PYTHON": "$VENV_DIR/bin/python" worked more reliably. There are some cases where I still ran into issues with @yemaney's approach.

Realistically they are the same when you call the devbox shell command from where the devbox.json is located which you would if you are calling it manually. I think the discrepancy may arise when using the VSCode extension for some reason or another.

@yemaney yemaney linked a pull request Jan 10, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working triage Issue needs triage
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(python plugin): set UV_PYTHON to python in .venv yemaney/devbox
3 participants
@MarkValadez @AlejandroGomezFrieiro @yemaney