Welcome to End Point’s blog

Ongoing observations by End Point people

Installing Python in Local Directory

On one of our client's Ubuntu 10.04 machines, I needed to upgrade Python from 2.6 to 2.7. Unfortunately, after installing Python 2.7 from apt the virtualenv, version 1.4.5, did not work correctly. This bug was fixed in a newer virtualenv version, however there were no Ubuntu packages available.

I thought about trying something else: why not install all the software locally in my home directory on the server? When virtualenv is used to create a new environment, it copies the Python executable in to the virtualenv directory.

First I install pythonbrew, which is great software for installing many different Python versions in a local directory.

$ curl -kL | bash

Then I activate pythonbrew with:

$ source "$HOME/.pythonbrew/etc/bashrc"

And install the Python version I want:

$ pythonbrew install 2.7.3

The installation took a couple of minutes. The script downloaded the tarball with the Python source code for the required version, compiled it and installed. It was writing all the information into a log file, which I was looking at by running the command below in another console:

$ tail -f $HOME/.pythonbrew/log/build.log

You can also add the following lines to ~/.bashrc to activate this python after starting new bash session.

[[ -s "$HOME/.pythonbrew/etc/bashrc" ]] && source $HOME/.pythonbrew/etc/bashrc
pythonbrew switch 2.7.3

I run the below command to activate the pythonbrew script:

$ source $HOME/.pythonbrew/etc/bashrc

The python version changed:

$ python --version
Python 2.6.5

$ source $HOME/.pythonbrew/etc/bashrc

$ python --version
Python 2.7.2

As you can see, Python from my local installation is used:

$ which python

The only thing left is to create the virtual environment for the new Python version. I use virtualenvwrapper for managing virtualenv, so the obvious way to create a new environment is:

$ mkvirtualenv --no-site-packages envname

Unfortunately, it creates an environment with the wrong Python version:

$ which python

$ python --version
Python 2.6.5

So let's try to tell virtualenvwrapper which Python file should be used:

$ deactivate 

$ rmvirtualenv envname
Removing envname...

$ mkvirtualenv --no-site-packages -p /home/szymon/.pythonbrew/pythons/Python-2.7.3/bin/python envname

Unfortunately this ended with an error:

Running virtualenv with interpreter /home/szymon/.pythonbrew/pythons/Python-2.7.3/bin/python
New python executable in envname/bin/python
Traceback (most recent call last):
  File "/home/szymon/.virtualenvs/envname/lib/python2.7/", line 67, in 
    import os
  File "/home/szymon/.virtualenvs/envname/lib/python2.7/", line 49, in 
    import posixpath as path
  File "/home/szymon/.virtualenvs/envname/lib/python2.7/", line 17, in 
    import warnings
ImportError: No module named warnings
ERROR: The executable envname/bin/python is not functioning
ERROR: It thinks sys.prefix is '/home/szymon/.virtualenvs' (should be '/home/szymon/.virtualenvs/envname')
ERROR: virtualenv is not compatible with this system or executable

The problem is that the virtualenv version, used by virtualenvwrapper, doesn't work with Python 2.7. As I wrote at the begining, there is no newer version available via apt. The solution is pretty simple. Let's just install the newer virtualenvwrapper and virtualenv version using pip.

$ pip install virtualenv
Requirement already satisfied: virtualenv in /usr/lib/pymodules/python2.6
Installing collected packages: virtualenv
Successfully installed virtualenv

As you can see, there is a problem. The problem is that there is used pip from system installation. There is no pip installed in my local Python 2.7 version. However there is easy_install:

$ which pip

$ which easy_install

So let's use it for installing virtualenv and virtualenvwrapper:

$ easy_install virtualenv virtualenvwrapper

I've checked the whole installation procedure once again, it turned out that there was some network error while downloading pip, but unfortunately I didn't notice the error. If everything is OK, then pip should be installed, and you should be able to install virtualenv using pip as well with:

$ pip install virtualenv virtualenvwrapper

Cool, let's check which version is installed:

$ which virtualenv

$ virtualenv --version

Before creating the brand new virtual environment, I have to activate the new virtualenvwrapper. I have the following line in my ~/.bashrc file:

source /usr/local/bin/

I just have to change it to the below line and login once again:

source /home/szymon/.pythonbrew/pythons/Python-2.7.3/bin/

Let's now create the virtual environment using the brand new Python version:

$ mkvirtualenv --no-site-packages -p $HOME/.pythonbrew/pythons/Python-2.7.3/bin/python envname

I want to use this environment each time I log into this server, so I've added this line to my ~/.bashrc:

workon envname

Let's check if it works. I've logged out and logged in to my account on this server before running the following commands:

$ which python

$ python --version
Python 2.7.3

Looks like everything is OK.

1 comment:

Pragmateek said...

Note that pythonbrew is no longer under active development.
You should use pyenv ( instead which provides the same features.
You may need the following prerequisites to install Python, e.g. 2.7.5, with pyenv install 2.7.5: libreadline-dev, libbz2-dev, libsqlite3-dev and libssl-dev.
By default pyenv is not really chatty, so if you want more info on what happens use the verbose option: "pyenv install --verbose 2.7.5".