Tech TLDR;

How to use Virtualenv in Python to Install Packages Locally

by

Table of Contents

There are two philosophies when it comes to package installation, global first and local first. Global meaning all applications that rely on a certain package have access to the same copy of the library that was installed once. Local means that each project has its own folder of dependencies installed specifically for this project and each library is installed many times. NPM, for example, uses local first strategy. Pip in Python uses global first strategy by default.

Both approaches have their benefits and drawbacks. In this post I wanted to show how to use a Python library called virtualenv to created isolated (local) python environments.

Installing virtualenv

As usual, it’s as simple as:

pip install virtualenv

Create a new virtual environment

Navigate to the folder where you want the project set up and run:

virtualenv any_name_you_want

Normally, I just name my python folder as venv so in my case I would run:

virtualenv venv

This will set up Python as well as pip that you can use locally.

Note that this command will install either Python 2 or Python 3 depending on your system set up. In my case it installs Python 3.

If I wanted to use Python 2, I can specify the path to Python version to use.

$ which python2
/usr/local/bin/python2 # <------- Use this

$ virtualenv -p /usr/local/bin/python2 venv
Code language: PHP (php)

For the purpose of this tutorial, let’s stick with default Python 3 installation.

Activating Virtual Environment

At this point we are still running Python in usual mode and running pip install will install a package globally.

We need to “switch” into the virtual environment by running the activation command:

source venv/bin/activate

At this point your session will be updated to point to your local python and pip from your venv folder.

You can verify this by running:

(venv) $ which python
/someproject/venv/bin/python

(venv) $ which pip
/someproject/venv/bin/pip

which will show that both Python and Pip are running locally.

Using the Virtual Environment

This is really it. You can now do your work in the virtual environment and anything installed with pip install will show up in your local venv folder.

For example let’s install Pandas.

(venv) $ pip install pandas
# ... pip magic

(venv) $ ls /someproject/venv/lib/python3.6/site-packages/ | grep pandas
pandas
pandas-0.23.4.dist-info
Code language: PHP (php)

Deactivating the Virtual Environment

When you are done with local environment, simply run deactivate in your shell to go back to the regular Python flow.

(venv) $ which python
/someproject/venv/bin/python

(venv) $ deactivate

$ which python
/usr/local/bin/python

Putting it all together

I have a repo where I show how to use Google Cloud API to convert speech to text. Let’s go ahead and clone it.

$ git clone git@github.com:akras14/speech-to-text.git
$ cd speech-to-text/
$ ls
.                .gitignore       requirements.txt transcripts
..               README.md        slow.py          venv
.git             fast.py          source

$ cat .gitignore
api-key.json
parts/
venv

Code language: PHP (php)

As you can see above, in my .gitignore file I have an entry for my venv folder, because I don’t want to check-in my local dependencies. Since there is no venv folder in this cloned repo, so let’s create it and source it.

$ virtualenv venv
$ source venv/bin/activate

At this point I am running local python and pip. Let me go ahead and install all dependencies from requirements.txt file.

$ pip install -r requirements.txt

After install is done I will see all needed packages installed in my /speech-to-text/venv/lib/python3.6/site-packages/ folder.

<br />(venv) $ cat requirements.txt
google-api-python-client==1.6.4
httplib2==0.10.3
oauth2client==4.1.2
pyasn1==0.4.2
pyasn1-modules==0.2.1
rsa==3.4.2
six==1.11.0
SpeechRecognition==3.8.1
tqdm==4.19.5
uritemplate==3.0.0


(venv) $ ls venv/lib/python3.6/site-packages/
SpeechRecognition-3.8.1.dist-info        oauth2client                             pyasn1_modules-0.2.1.dist-info           tqdm
__pycache__                              oauth2client-4.1.2.dist-info             rsa                                      tqdm-4.19.5.dist-info
apiclient                                pip                                      rsa-3.4.2.dist-info                      uritemplate
easy_install.py                          pip-18.0.dist-info                       setuptools                               uritemplate-3.0.0.dist-info
google_api_python_client-1.6.4.dist-info pkg_resources                            setuptools-40.1.0.dist-info              wheel
googleapiclient                          pyasn1                                   six-1.11.0.dist-info                     wheel-0.31.1.dist-info
httplib2                                 pyasn1-0.4.2.dist-info                   six.py
httplib2-0.10.3.dist-info                pyasn1_modules                           speech_recognition
Code language: HTML, XML (xml)

As a side note, requirements.txt is a pip convention for storing dependencies. Kind of like package.json file in NPM.

pip has a command called freeze which will list all dependencies for the project.

$ pip freeze
google-api-python-client==1.6.4
httplib2==0.10.3
oauth2client==4.1.2
pyasn1==0.4.2
pyasn1-modules==0.2.1
rsa==3.4.2
six==1.11.0
SpeechRecognition==3.8.1
tqdm==4.19.5
uritemplate==3.0.0

To record project dependancies we just have to redirect result of pip freeze into a txt file.

pip freeze > requirements.txt
Code language: CSS (css)

Checking this file in, will make sure that the next person using this project will have the same dependencies as we did.

In any case, now that our dependancies are installed, we are ready to run the repo. Once we are done, we can de-activate it or simply exit this shell session.

deactivate

That’s really it. In true spirit of Python – using virtualenv is very simple and effective.