Python Language pip: PyPI Package Manager Installing packages not yet on pip as wheels
Example
Many, pure python, packages are not yet available on the Python Package Index as wheels but still install fine. However, some packages on Windows give the dreaded vcvarsall.bat not found error.
The problem is that the package that you are trying to install contains a C or C++ extension and is not currently available as a pre-built wheel from the python package index, pypi, and on windows you do not have the tool chain needed to build such items.
The simplest answer is to go to Christoph Gohlke's excellent site and locate the appropriate version of the libraries that you need. By appropriate in the package name a -cpNN- has to match your version of python, i.e. if you are using windows 32 bit python even on win64 the name must include -win32- and if using the 64 bit python it must include -win_amd64- and then the python version must match, i.e. for Python 34 the filename must include -cp34-, etc. this is basically the magic that pip does for you on the pypi site.
Alternatively, you need to get the appropriate windows development kit for the version of python that you are using, the headers for any library that the package you are trying to build interfaces to, possibly the python headers for the version of python, etc.
Python 2.7 used Visual Studio 2008, Python 3.3 and 3.4 used Visual Studio 2010, and Python 3.5+ uses Visual Studio 2015.
- Install “Visual C++ Compiler Package for Python 2.7”, which is available from Microsoft’s website or
- Install “Windows SDK for Windows 7 and .NET Framework 4” (v7.1), which is available from Microsoft’s website or
- Install Visual Studio 2015 Community Edition, (or any later version, when these are released), ensuring you select the options to install C & C++ support no longer the default -I am told that this can take up to 8 hours to download and install so make sure that those options are set on the first try.
Then you may need to locate the header files, at the matching revision for any libraries that your desired package links to and download those to an appropriate locations.
Finally you can let pip do your build - of course if the package has dependencies that you don't yet have you may also need to find the header files for them as well.
Alternatives: It is also worth looking out, both on pypi or Christop's site, for any slightly earlier version of the package that you are looking for that is either pure python or pre-built for your platform and python version and possibly using those, if found, until your package does become available. Likewise if you are using the very latest version of python you may find that it takes the package maintainers a little time to catch up so for projects that really need a specific package you may have to use a slightly older python for the moment. You can also check the packages source site to see if there is a forked version that is available pre-built or as pure python and searching for alternative packages that provide the functionality that you require but are available - one example that springs to mind is the Pillow, actively maintained, drop in replacement for PIL currently not updated in 6 years and not available for python 3.
Afterword, I would encourage anybody who is having this problem to go to the bug tracker for the package and add to, or raise if there isn't one already, a ticket politely requesting that the package maintainers provide a wheel on pypi for your specific combination of platform and python, if this is done then normally things will get better with time, some package maintainers don't realise that they have missed a given combination that people may be using.
Note on Installing Pre-Releases
Pip follows the rules of Semantic Versioning and by default prefers released packages over pre-releases. So if a given package has been released as V0.98 and there is also a release candidate V1.0-rc1 the default behaviour of pip install will be to install V0.98 - if you wish to install the release candidate, you are advised to test in a virtual environment first, you can enable do so with --pip install --pre package-name or --pip install --pre --upgrade package-name. In many cases pre-releases or release candidates may not have wheels built for all platform & version combinations so you are more likely to encounter the issues above.
Note on Installing Development Versions
You can also use pip to install development versions of packages from github and other locations, since such code is in flux it is very unlikely to have wheels built for it, so any impure packages will require the presence of the build tools, and they may be broken at any time so the user is strongly encouraged to only install such packages in a virtual environment.
Three options exist for such installations:
- Download compressed snapshot, most online version control systems have the option to download a compressed snapshot of the code. This can be downloaded manually and then installed with
pip installpath/to/downloaded/file note that for most compression formats pip will handle unpacking to a cache area, etc. - Let pip handle the download & install for you with:
pip installURL/of/package/repository - you may also need to use the--trusted-host,--client-certand/or--proxyflags for this to work correctly, especially in a corporate environment. e.g:
> py -3.5-32 -m venv demo-pip
> demo-pip\Scripts\activate.bat
> python -m pip install -U pip
Collecting pip
Using cached pip-9.0.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 8.1.1
Uninstalling pip-8.1.1:
Successfully uninstalled pip-8.1.1
Successfully installed pip-9.0.1
> pip install git+https://github.com/sphinx-doc/sphinx/
Collecting git+https://github.com/sphinx-doc/sphinx/
Cloning https://github.com/sphinx-doc/sphinx/ to c:\users\steve-~1\appdata\local\temp\pip-04yn9hpp-build
Collecting six>=1.5 (from Sphinx==1.7.dev20170506)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting Jinja2>=2.3 (from Sphinx==1.7.dev20170506)
Using cached Jinja2-2.9.6-py2.py3-none-any.whl
Collecting Pygments>=2.0 (from Sphinx==1.7.dev20170506)
Using cached Pygments-2.2.0-py2.py3-none-any.whl
Collecting docutils>=0.11 (from Sphinx==1.7.dev20170506)
Using cached docutils-0.13.1-py3-none-any.whl
Collecting snowballstemmer>=1.1 (from Sphinx==1.7.dev20170506)
Using cached snowballstemmer-1.2.1-py2.py3-none-any.whl
Collecting babel!=2.0,>=1.3 (from Sphinx==1.7.dev20170506)
Using cached Babel-2.4.0-py2.py3-none-any.whl
Collecting alabaster<0.8,>=0.7 (from Sphinx==1.7.dev20170506)
Using cached alabaster-0.7.10-py2.py3-none-any.whl
Collecting imagesize (from Sphinx==1.7.dev20170506)
Using cached imagesize-0.7.1-py2.py3-none-any.whl
Collecting requests>=2.0.0 (from Sphinx==1.7.dev20170506)
Using cached requests-2.13.0-py2.py3-none-any.whl
Collecting typing (from Sphinx==1.7.dev20170506)
Using cached typing-3.6.1.tar.gz
Requirement already satisfied: setuptools in f:\toolbuild\temp\demo-pip\lib\site-packages (from Sphinx==1.7.dev20170506)
Collecting sphinxcontrib-websupport (from Sphinx==1.7.dev20170506)
Downloading sphinxcontrib_websupport-1.0.0-py2.py3-none-any.whl
Collecting colorama>=0.3.5 (from Sphinx==1.7.dev20170506)
Using cached colorama-0.3.9-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.3->Sphinx==1.7.dev20170506)
Using cached MarkupSafe-1.0.tar.gz
Collecting pytz>=0a (from babel!=2.0,>=1.3->Sphinx==1.7.dev20170506)
Using cached pytz-2017.2-py2.py3-none-any.whl
Collecting sqlalchemy>=0.9 (from sphinxcontrib-websupport->Sphinx==1.7.dev20170506)
Downloading SQLAlchemy-1.1.9.tar.gz (5.2MB)
100% |################################| 5.2MB 220kB/s
Collecting whoosh>=2.0 (from sphinxcontrib-websupport->Sphinx==1.7.dev20170506)
Downloading Whoosh-2.7.4-py2.py3-none-any.whl (468kB)
100% |################################| 471kB 1.1MB/s
Installing collected packages: six, MarkupSafe, Jinja2, Pygments, docutils, snowballstemmer, pytz, babel, alabaster, imagesize, requests, typing, sqlalchemy, whoosh, sphinxcontrib-websupport, colorama, Sphinx
Running setup.py install for MarkupSafe ... done
Running setup.py install for typing ... done
Running setup.py install for sqlalchemy ... done
Running setup.py install for Sphinx ... done
Successfully installed Jinja2-2.9.6 MarkupSafe-1.0 Pygments-2.2.0 Sphinx-1.7.dev20170506 alabaster-0.7.10 babel-2.4.0 colorama-0.3.9 docutils-0.13.1 imagesize-0.7.1 pytz-2017.2 requests-2.13.0 six-1.10.0 snowballstemmer-1.2.1 sphinxcontrib-websupport-1.0.0 sqlalchemy-1.1.9 typing-3.6.1 whoosh-2.7.4
Note the git+ prefix to the URL.
- Clone the repository using
git,mercurialor other acceptable tool, preferably a DVCS tool, and usepip installpath/to/cloned/repo - this will both process any requires.text file and perform the build and setup steps, you can manually change directory to your cloned repository and runpip install -r requires.txtand thenpython setup.py installto get the same effect. The big advantages of this approach is that while the initial clone operation may take longer than the snapshot download you can update to the latest with, in the case of git:git pull origin masterand if the current version contains errors you can usepip uninstallpackage-name then usegit checkoutcommands to move back through the repository history to earlier version(s) and re-try.
