23

I found that in different folders, sometimes pip install will build wheel which takes a lot of time, while sometimes it doesn't. I'm not sure why and how to control that.

My command: bin/python -m pip install -r ../requirements.txt (due to the !# shebang line-length limitation, so I don't use pip directly)

The output without building a wheel (just takes a few seconds):

Collecting numpy==1.10.4 (from -r ../requirements.txt (line 1))
Installing collected packages: numpy
Successfully installed numpy-1.10.4

The output with building wheel (take at least 2 minutes)

Collecting numpy==1.10.4 (from -r ../requirements.txt (line 1))
  Downloading numpy-1.10.4.tar.gz (4.1MB)
    100% |████████████████████████████████| 4.1MB 92kB/s
Building wheels for collected packages: numpy
  Running setup.py bdist_wheel for numpy ... done
  Stored in directory: /root/.cache/pip/wheels/66/f5/d7/f6ddd78b61037fcb51a3e32c9cd276e292343cdd62d5384efd
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.10.4

The contents of requirements.txt:

numpy==1.10.4
Rob Bednark
  • 22,937
  • 20
  • 77
  • 112
zjffdu
  • 22,452
  • 44
  • 99
  • 141

3 Answers3

19

Today I encountered a problem where a package wasn't being installed properly because it turns out that its build process generates incorrect wheel packages, even though direct installation works just fine.

I did a bit of poking around, and it turns out that as of now (pip == 8.1.2), there isn't a direct way to control whether or not pip will try to build a wheel out of a given package. I found the relevant source code, and apparently, the wheel build process is used if and only if:

  • the wheel module is importable
  • a cache directory is in use

As a result of that logic, one can indirectly disable pip's use of wheel-based builds by passing --no-cache-dir on the install command line.

0 _
  • 9,294
  • 10
  • 71
  • 105
Simon Ruggier
  • 381
  • 2
  • 5
  • 2
    Looks like this logic has changed quite a bit (now at ``pip==19``). Code can now be traced to https://github.com/pypa/pip/blob/cf722df4f9d7f8ee2d62e53ef2354ea65e932170/src/pip/_internal/wheel_builder.py#L48. Unfortunately, there doesn't appear to be a convenient flag to turn off the wheel building functionality (other than uninstalling wheel) whilst keeping wheel installations for packages already with wheels. – pelson Jan 14 '20 at 12:35
  • 5
    Actually, you can be specific about which packages shouldn't generate wheels by setting the ``--no-binary={package_name}`` apparently... – pelson Jan 14 '20 at 12:47
  • 1
    Pip also has the `--only-binary` and `--prefer-binary` arguments. – cowlinator Feb 10 '21 at 20:06
7

This depends on whether your package is a pure python package (without the need to compile anything, just copy the files somewhere) or a package which also includes c source code (in which case a compilation is necessary and a compiler is called and executed, which takes longer).

http://pythonwheels.com/

You may also want to have a look at the wheel docu:

http://wheel.readthedocs.org/en/latest/

tfv
  • 5,704
  • 3
  • 31
  • 62
  • 12
    Whether or not `numpy` is a pure python package would describe whether *it* would cause python to build a wheel, but that doesn't explain why sometimes it builds the wheel and sometimes it doesn't, as the question shows. – Cory Klein Sep 18 '17 at 19:56
4

I got the answer, it is just the first time that the wheel will be build, after that, it will read from cache

zjffdu
  • 22,452
  • 44
  • 99
  • 141