A Python virtual environment is an important tool for developers to separate project dependencies in isolated environments. If working locally on several Python projects, developers may need to use different Python versions on each.
By doing this, they will be able to install different packages and have different versions of the named packages for each environment.
However, this can generate a lot of compatibility issues unless you manage them correctly using virtual environments. There are many ways to do this and I will highlight two tools to do so: Pyenv and Pipenv.
Python Virtual Environments
In the following image, you can see an abstract example of different Python projects containing the different combinations of packages that you can install:
Each big colored box represents a Python virtual environment. It's common for certain package versions to be supported only by determined Python versions. Also, you may need a determined version of a package in one project and another version for others.
As well as this, a determined version of a package can generate conflicts with packages or Python versions.
Many combinations of compatibility issues can be found but by having those isolated environments you can avoid them. You will be able to install, uninstall, update, etc in one virtual environment, without affecting the rest of them.
As mentioned, there are many tools to do this, and I will talk about the ones I find easy to understand and use.
Pyenv
Pyenv is a tool used to manage different Python versions. With this tool you can:
- Install several Python versions.
- Set/change the global(default) Python version in your computer.
- Set/change a Python version locally for a project.
- Create and manage virtual environments.
Installation
On macOS, you can install this tool using Homebrew:
[.c-inline-code]brew install pyenv
[.c-inline-code]
To install it on other platforms, you can watch this section in the GitHub docs. After the installation, edit your used shell startup script ([.c-inline-code].bashrc[.c-inline-code], [.c-inline-code].zshrc[.c-inline-code], [.c-inline-code].bash_profile[.c-inline-code], ...) adding the following line:
CODE: https://gist.github.com/brunomichetti/68d68d339870ec7fffed7494a4d141db.js
[.c-inline-code][.c-inline-code]
Then reload your shell startup script. An example if you have [.c-inline-code].bashrc[.c-inline-code]:
[.c-inline-code][.c-inline-code]cd[.c-inline-code]
[.c-inline-code]source .bashrc[.c-inline-code][.c-inline-code]
That's it! You have installed Pyenv successfully.
Managing Python Versions
Now let's take a look at the possible Python versions that you can install through Pyenv, executing:
[.c-inline-code]pyenv install --list[.c-inline-code]
You will see a large list of Python versions. You can pick one to install it, let's say 3.9.1:
[.c-inline-code]pyenv install 3.9.1[.c-inline-code]
You can run that command with any version that you want to have installed on your computer. To see all the installed versions, just run:
[.c-inline-code]pyenv versions[.c-inline-code]
Also, you can set the global version that you want to use in your system, this will be the default one. For example, if you have already installed 3.8.8, you can set it as the global version running:
[.c-inline-code]pyenv global 3.8.8[.c-inline-code]
You can change that, of course, by executing the command again with the desired installed version. If you want to check your global version, just run:
[.c-inline-code]python -V[.c-inline-code]
With Pyenv, you can have a global and a local version for each project. To set a local version, go to the project folder in the terminal:
[.c-inline-code]cd ~/path/to/the/project/folder[.c-inline-code]
And set the local version, for example, 3.9.1:
[.c-inline-code]pyenv local 3.9.1[.c-inline-code]
This will create a [.c-inline-code].python-version[.c-inline-code] file in the folder indicating the current local Python version for the project. Also, if you run [.c-inline-code]python -V[.c-inline-code] in that folder, you will see the local version, and not the global one.
So, to sum up, you have the following useful commands at your disposal:
- [.c-inline-code]pyenv install --list[.c-inline-code] to see the available Python versions you can install.
- [.c-inline-code]pyenv versions[.c-inline-code] to see the installed Python versions.
- [.c-inline-code]pyenv global <Python version>[.c-inline-code] to set an installed Python version as global.
- [.c-inline-code]pyenv local <Python version>[.c-inline-code] to set an installed Python version for a given project folder.
- [.c-inline-code]pyenv uninstall <Python version>[.c-inline-code] to uninstall an already installed Python version.
Pyenv also allows us to manage environments with a plugin called pyenv-virtualenv. However, I prefer to use Pipenv. Let's take a look.
Pipenv
Pipenv is a tool used to manage the required packages in a Python project. With this tool you can:
- Automatically create or delete a virtual environment.
- Install, uninstall and update packages maintaining the desired versions.
- Set up very quickly a working Python environment.
- Have a list of the installed packages and the corresponding version.
If you use Pipenv in your projects, you will have two special files in the root folder, both generated and changed automatically. Those are:
- [.c-inline-code]Pipfile[.c-inline-code]: A file that specifies the installed packages in your virtual environment for development and execution.
- [.c-inline-code]Pipfile.lock[.c-inline-code]: A file that specifies which versions of the installed packages (detailed in the Pipfile) has to be used. Also has another important metadata.
With Pipenv, is very easy to work on projects with other developers using the same packages and versions as it provides the files containing all that information. Developers must simply run a command to have the same environment in their own computer.
Installation
On macOS, you can install this tool using Homebrew:
[.c-inline-code]brew install pipenv[.c-inline-code]
And that's it. You're ready to start! To install it on other platforms, check out this section in the pipenv docs.
Creating Virtual Environments
Now, let's look at how to create a virtual environment for a Python project. First, go to the project folder:
[.c-inline-code]cd ~/path/to/the/project/folder[.c-inline-code]
And then execute:
[.c-inline-code]pipenv install[.c-inline-code]
This will create a virtual environment and:
- If you already have a Pipfile and a Pipfile.lock: it will also install all the specified packages on them.
- If you don't have a Pipfile and a Pipfile.lock: it will generate them for that environment.
This is how a Pipfile recently created looks like:
CODE: https://gist.github.com/brunomichetti/fd72edde62328001d741b62882a3cf34.js
[.c-inline-code][.c-inline-code]
Under the [.c-inline-code][packages][.c-inline-code] section, you will see the installed packages for execution - the ones that your Python program will need to be executed correctly.
Under the [.c-inline-code][dev-packages][.c-inline-code] section, you will see the packages that are needed only for development. Also, you can see in that example that the used Python version is [.c-inline-code]3.7[.c-inline-code].
If you use Pyenv, the [.c-inline-code]pipenv install[.c-inline-code] command will use the global Python version of your system. But as you know, you can specify a local Python version in your project. For example:
[.c-inline-code]pyenv local 3.8.8[.c-inline-code]
And, if you want to use that local Python version, you can indicate that to Pipenv by executing:
[.c-inline-code]pipenv install --python 3.8.[.c-inline-code]
Now, what if you want to delete the created virtual environment to create another one? You can do so by running:
[.c-inline-code]pipenv --rm[.c-inline-code]
Another important thing to remember is to activate the created virtual environment, which you do by running:
[.c-inline-code]pipenv shell[.c-inline-code]
With this command, you "have entered" your virtual environment. In your terminal, you can see if you have already activated the virtual environment, if you see the name between brackets at the beginning of the line, as follows:
There you can see that after the [.c-inline-code]pipenv shell[.c-inline-code] command has been executed, the name of the virtual environment appeared between brackets at the beginning of the line.
To deactivate the virtual environment, just run:
[.c-inline-code]exit[.c-inline-code]
Managing Packages
Ok, so you now know how to create, delete, activate and deactivate a virtual environment. Now let's see how to manage the packages inside them.
First, don't forget to activate the virtual environment. To install a package, for example django-drip-campaings, you can execute:
[.c-inline-code]pipenv install django-drip-campaigns[.c-inline-code]
This command will:
- Install the latest available version of that package.
- Automatically update the Pipfile file adding the package.
- Automatically update the Pipfile.lock file adding the package version and some other important metadata.
Now the Pipfile file looks like this:
CODE: https://gist.github.com/brunomichetti/e8328cddc93bde60d4623d660a6d6faf.js
[.c-inline-code][.c-inline-code]
As you can see, the package was installed under the [.c-inline-code][packages][.c-inline-code] section. As previously mentioned, in the [.c-inline-code][dev-packages][.c-inline-code] section you only install what’s needed for development.
For example, you want to have flake8 in your project, which is only required by the development process. To install it under the [.c-inline-code][dev-packages][.c-inline-code] section, execute:
[.c-inline-code]pipenv install flake8 --dev[.c-inline-code]
After that, you will see something like this in the Pipfile file:
[.c-inline-code][packages]
django-drip-campaigns = "*"
[dev-packages]
flake8 = "*"
[.c-inline-code]
Also, what if you need a specific version of a package? For example - Django, not the latest but the 2.2 version. Then run:
[.c-inline-code]pipenv install django==2.2[.c-inline-code]
Now the Pipfile is updated and will resemble something like:
CODE: https://gist.github.com/brunomichetti/59b0125a1026fd8210d96eb6a0b5d0af.js
In the [.c-inline-code]Django[.c-inline-code] package line, you don't see the "*", you instead see the specific version indicated in the command. If you need to uninstall a package, for example [.c-inline-code]django-drip-campaigns[.c-inline-code], just run:
[.c-inline-code]pipenv uninstall django-drip-campaigns[.c-inline-code]
That command will uninstall the package and remove the information from the Pipfile and Pipfile.lock files. Now suppose you need the latest version of [.c-inline-code]Django[.c-inline-code] in your project (not the 2.2 version).
To update a package to the latest version, you have to:
- Change the [.c-inline-code]django = "==2.2"[.c-inline-code] line in the Pipfile by [.c-inline-code]django = "*"[.c-inline-code].
- Then run:
[.c-inline-code]pipenv update django[.c-inline-code]
Alternatively, you can uninstall [.c-inline-code]Django[.c-inline-code] and then install it again without specifying the version. You can repeat those steps to downgrade a package version instead of upgrading it.
You can do a lot of things with Pipenv, but the previously mentioned throughout are the main features. Pretty easy, right?
Now let's see a summary of the mentioned commands provided by Pipenv:
- [.c-inline-code]pipenv install[.c-inline-code] to create a virtual environment.
- [.c-inline-code]pipenv install --python <Python version>[.c-inline-code] to create a virtual environment indicating the desired Python version (that you have installed using Pyenv).
- [.c-inline-code]pipenv --rm[.c-inline-code] to delete the current virtual environment.
- [.c-inline-code]pipenv shell[.c-inline-code] to activate the created virtual environment.
- [.c-inline-code]exit[.c-inline-code] to deactivate an already activated virtual environment.
Now let's take a look at the summary of the commands after the virtual environment has been activated:
- [.c-inline-code]pipenv install <package name>[.c-inline-code] to install the latest version of the package under the [.c-inline-code][packages][.c-inline-code] section.
- [.c-inline-code]pipenv install <package name>==<package version>[.c-inline-code] to install a specified version of a package, under the [.c-inline-code][packages][.c-inline-code] section.
- [.c-inline-code][.c-inline-code]pipenv update <package name>[.c-inline-code][.c-inline-code] to update a version of a package(upgrade or downgrade) to the one that you have previously specified in the Pipfile.
- Adding [.c-inline-code]--dev[.c-inline-code] to the previous commands, Pipenv will do the same but under the [.c-inline-code][dev-packages][.c-inline-code] section.
- [.c-inline-code]pipenv uninstall <package name>[.c-inline-code] to uninstall a package.
Summary
As highlighted throughout, I have presented two different tools for managing Python projects using virtual environments. There are many ways to do this, but I showcased the two I believe to be the most effective.
As tools, Pyenv and Pipenv are tidy and maintainable as well as easy to understand and use. Pyenv is used to manage different Python versions, whereas Pipenv is used to manage Python packages.
I really like using both and I hope you are encouraged to try these tools for Python development. Thank you for reading and stay tuned for more useful content.
